Development of Methods for Selection and Evaluation in Undergraduate Medical Education

The purposes of this study were (1) to develop and validate measures of diagnostic problem solving that might be useful in the selection of medical students and in evaluating performance during training, and (2) to improve our understanding of the cognitive processes involved in such problem solving. The study involved three kinds of measures. (1) Criterion tests were paper‐and‐pencil simulations of the salient aspects of medical diagnostic situations. Three such problems used a free‐response format that required examinees to think of ideas for themselves, while three employed a structured‐response (expanded multiple‐choice) format. (2) Experimental selection tests were paper‐and‐pencil simulations of situations that present problems of a nonmedical nature and that could presumably be solved by methods like those used for medical problems. Both free‐response and structured‐response problems were included. (3) Process measures were selected from existing test batteries to measure cognitive abilities and personality characteristics thought to be involved in carrying out problem‐solving activities. All three kinds of measures were administered to 122 fourth‐year medical students. The selection tests and the process variables were given to 97 first‐year students. Finally, a small number (30) of advanced residents took the criterion tests only. Data from student records, such as MCAT and National Board scores, were also obtained as appropriate.