Key points are not available for this paper at this time.
Effectiveness of Heuristic Evaluation (HE) has severely been challenged. The present study investigated two strategies for improving the effectiveness of HE: Selection of usability guidelines (Gerhardt-Powals' cognitive engineering principles vs. Nielsen's heuristics); Provision of support and training. We also examined various strategies for estimating the effectiveness of HE. An empirical study with a balanced within- and between subject 2x2 factorial design involving 18 novice evaluators was conducted. 290 non-consolidated usability problems (UPs) identified in 35 HE test sessions were evaluated against 88 actual UPs found in the user tests (n = 17) performed earlier. Nielsen's heuristics were proved to be more effective in enabling the evaluators to uncover significantly more UPs, though in general the effectiveness of HE was low. Implications for future work are drawn.
Law et al. (Sat,) studied this question.