Psychometric evaluation of an after-scenario questionnaire for computer usability studies

A three-item after-scenario questionnaire was used in three related usability tests in different areas of the United States. The studies had eight scenarios in common. After participants finished a scenario, they completed the After-Scenario Questionnaire (the ASQ). A factor analysis of the responses to the ASQ items revealed that an eight-factor solution explained 94 percent of the variability of the 24 (eight scenarios by three items per scenario) items. The varimax-rotated factor pattern showed that these eight factors were clearly associated with the eight scenarios. The benefit of this research to system designers is that this three-item questionnaire has acceptable psychometric properties of reliability, sensitivity, and concurrent validity, and may be used with confidence in other, similar usability studies.