Subjective evaluation

Discussion articles

• Subjective evaluation
  Subjective assessment tells the evaluator how the users feel about the software being tested. This is distinct from how efficiently or effectively they perform with the software. The usual method of assessment is to used a standardised opinion questionnaire to avoid criticisms of subjectivity.

Evaluation tools

• A comparison of questionnaire's for assessing website usability (PDF)
  Five questionnaires for assessing the usability of a website were compared in a study with 123 participants. The questionnaires studied were SUS, QUIS, CSUQ, a variant of Microsoft’s Product Reaction Cards, and one that we have used in our Usability Lab for several years. Each participant performed two tasks on each of two websites: finance.yahoo.com and kiplinger.com. All five questionnaires revealed that one site was significantly preferred over the other. The data were analyzed to determine what the results would have been at different sample sizes from 6 to 14. At a sample size of 6, only 30-40% of the samples would have identified that one of the sites was significantly preferred. Most of the data reach an apparent asymptote at a sample size of 12, where two of the questionnaires (SUS and CSUQ) yielded the same conclusion as the full dataset at least 90% of the time.

• QUIS: The Questionnaire for User Interaction Satisfaction
  Subjective evaluation is an important component in the evaluation of workstation usability. We have developed and standardised a general user evaluation instrument for interactive computer systems. The methods of psychological test construction were applied in order to ensure proper construct and empirical validity of the items and to assess their reliability. A hierarchical approach was taken in which overall usability was divided into subcomponents which constituted independent psychometric scales. For example, subcomponents include character readability, usefulness of online help, and meaningfulness of error messages. Evaluation on these scales is assessed by user ratings of specific system attributes such as character definition, contrast, font, and spacing for the scale of character readability.

• SUMI
  The Software Usability Measurement Inventory is a rigorously tested and proven method of measuring software quality from the end user's point of view. SUMI is a consistent method for assessing the quality of use of a software product or prototype, and can assist with the detection of usability flaws before a product is shipped.

• WAMMI
  WAMMI is an evaluation tool for web sites. It is based on a questionnaire that your visitors fill out, and which gives you a measure of how easy to use they think your web site is. WAMMI is no ordinary questionnaire. The questions have been carefully selected and refined to ascertain users' subjective rating of the ease of use of a web site - iteratively improved and tested on numerous web sites