Evaluating the Predictive Validity of MCAT Scores across Diverse Applicant Groups

PURPOSE: To examine the predictive validity of MCAT scores, alone and in combination with other preadmission data, for medical students grouped by race/ethnicity and sex. METHOD: This study included two samples: 1,109 students who entered in 1992 any of the 14 medical schools participating in the MCAT Predictive Validity Study; and all 11,279 students who entered medical school in 1992 and took the USMLE Step 1 in June 1994. Criterion measures included each student's cumulative GPA in the first two years of medical school and his or her pass/fail status on Step 1. Differential predictive validity was examined by comparing prediction errors across racial/ethnic and sex groups. For cumulative GPA; residuals were compared, and for Step 1, classification errors were studied. RESULTS: The patterns of prediction errors observed across the groups indicated that, on average, (1) no difference between the sexes in prediction errors was evident; (2) performances of the three racial/ethnic minority groups tended to be overpredicted, with significant findings for Asians and Hispanics; and (3) Caucasians' performance tended to be underpredicted, although the magnitude of this underprediction was quite small. When USMLE Step 1 scores were the criterion for success in medical school, the majority of errors were overprediction errors. CONCLUSION: The authors caution that although MCAT scores, alone and in combination with undergraduate GPA, are good predictors of medical school performance, they are not perfect. The authors encourage future research exploring additional predictor variables, such as diligence, motivation, communication skills, study habits, and other relevant characteristics. Similarly, they indicate that high grades and Step 1 scores are not the only indicators of success in the medical profession and call for studies examining other important qualities, such as integrity, interpersonal skills, capacity for caring, willingness to commit to lifelong learning, and desire to serve in underserved areas.