The Effects of Judgment Strategy and Prompting on Using Rule-Based Expert Systems for Knowledge Transfer.

Abstract This study examines how alternative designs of an expert system affect knowledge transfer to users. Researchers have long speculated that expert systems could be used for knowledge transfer (Biggs et al. 1987; Borthick and West 1987; Gal and Steinbart 1987). A number of studies have compared expert system use to unaided practice (Dorr et al. 1988; Eining and Dorr 1991; Fedorowicz et al. 1992; Murphy 1990; Murphy and Phillips 1991). Such comparisons imply that expert systems are an alternative to conventional training methods. Judgment skills, however, are developed not only through formal training but also through practical experience. Increasingly, that experience involves the use of an expert system (Brown 1991; Brown and Murphy 1990; Meservy et al. 1992). These systems contain valuable knowledge, and organizations can reap significant benefits if the knowledge could be transferred to less experienced decision makers as they use the system to make decisions. Studies have found that decision performance is improved when the expert system is designed to match both the user's understanding of the task and the characteristics of the task (Lamberti and Newsome 1989; Lamberti and Wallace 1990). An important question, therefore, is how alternative designs of an expert system affect knowledge transfer to users. An experiment was conducted in which subjects used different versions of the same expert system to practice evaluating internal controls. Two design features were varied: prompting and judgment strategy. One group of subjects used an expert system in which the rules being followed were displayed during the consultation; the remaining subjects not only saw the rules, but were also periodically prompted to think about the judgment strategy those rules represented. One group of subjects used an expert system that approached the evaluation of internal controls in terms of the stage of the processing cycle during which the control was applied; the other group used an expert system that focused on the objectives of the controls. Both prompting and judgment strategy affected knowledge transfer. Subjects who were prompted to think about the meaning of the rules generally exhibited greater learning than did subjects who were not prompted to think about them. Subjects who used an expert system that followed the strategy of asking questions according to control objectives generally also exhibited greater knowledge transfer.