Usability Evaluation and Prototype Fidelity: Users and Usability Professionals

This study investigated the effect of prototype fidelity on the information obtained from a usability test with potential users of a software product. Users engaged in a series of structured tasks using one of three prototypes which emulated a Microsoft® Windows 95™ based library search computer system and which differed only in their fidelity (low = paper; medium = screen shots, high = interactive Visual Basic prototype). Results indicated there were no significant differences as a function of prototype in the number and severity of problems encountered nor in the subjective evaluation of the product. More importantly, there was a high degree of commonality in the specific problems uncovered by users using the three prototypes. Other noteworthy results were that: (1) usability professionals (through informal heuristics analysis) and users differed in the incidence and commonality of problems identified, and (2) usability professionals did not agree in their ratings of problem severity.