Simulated patients in assessing consultation skills of trainees in general practice vocational training: a validity study

Summary. Although simulated patients are increasingly used in medical education, little research has been carried out on their validity. Validity in this case defines the relationship between performance with a simulated patient and performance with a real patient. One of the objectives of this study was to determine the validity of the use of simulated patients in assessing the consultation skills of trainees in vocational training at the Department of General Practice, University of Utrecht, The Netherlands. A check‐list with a rating scale was used to assess the consultation skills of trainees at the department with simulated patients as well as in their training practices with real patients. The simulated and the selected practice cases were patients with complex multi‐conditional problems like low back pain, headache and chest pain. The consultation skills were subdivided into four groups: the patient‐centred approach, the non‐somatic approach, communication skills and interpersonal skills. The measurement of skills, in particular of consultation skills, is very difficult. A description is given of the way the research group solved this problem. The analysis was performed by determining the sensitivity and predictive value of the assessment of a simulated encounter with a routine practice encounter. A difference existed in the assessed level of consultation skills in the simulated encounter compared to the level in the training practice. In simulation the level of consultation skills was higher than in day‐to‐day practice. This difference can reflect the difference between competence and performance. Competence is defined as what a doctor is capable of doing and performance as what a doctor actually does in day‐to‐day practice. The assessment of behaviour in simulation reflected well those trainees who also performed inadequately with practice cases. The predictive value of inadequate performance in simulation is high. For feedback purposes this is an important issue. Because of the difference between competence (simulation) and performance (real practice), the predictive value of the behaviour rated adequate in simulation is not so high. Considering the difference between competence and performance, actor simulation is a valid method to assess consultation skills of trainees.