Abstract This paper provides evidence on the problem-solving behavior of expert and novice auditors in the context of a control risk assessment task. Understanding the problem-solving behavior of auditors is necessary in order to evaluate how well auditors conduct the problem-solving process and where it can be improved. When the correctness of a decision cannot be determined, experts' behavior can be used as a benchmark. Also, auditing firms allocate considerable resources to train auditors and to develop decision aids and expert systems that will help transfer experts' knowledge to novices. Understanding differences between expert and novice problem-solving behavior provides a basis for decision evaluation, training, and improvement. However, tracing auditors' problem-solving behavior is not an easy task. This study employs a process-tracing methodology comparable to protocol analysis that helps overcome the constraints of sample size, reliability, and costs associated with verbal protocol analysis. This is done by using a computer-controlled information retrieval system (CIRS) that traces the subjects' information search behavior. Three dimensions of information search behavior are examined: information search strategy, information acquisition, and information search duration. Twenty-four computer audit specialists (experts) and 28 non-specialist auditors (novices) participated in the study. The audit case implemented on the CIRS involves a computerized accounting system for the revenue cycle of a hardware distributor. The case includes 201 information items on the background of the client, the general computer controls, the application controls, and the audit decisions. Use of the CIRS instead of verbal protocol analysis proved successful in enhancing reliability, reducing costs of coding, and expanding sample size. In general, experts and novices exhibited different information search behavior. Experts exhibited a more global search pattern guided by an overall planning strategy. They also acquired significantly less information on general computer controls but attached more importance to these controls, indicating that they might acquire information more efficiently. Experts also acquired significantly fewer redundant computer controls than novices. Finally, experts were more efficient than novices in terms of information search time, requiring significantly less time to perform the task. These findings show that understanding differences between experts and novices provides a good basis for the evaluation of auditors' problem-solving and suggest areas where audit decision-making needs to be improved.
Bödard et al. (Wed,) studied this question.