Key points are not available for this paper at this time.
The annual selection of new students is one of the most important activities of medical school faculty. They face the challenge of selecting those who can perform well not only in the preclinical years, but also in the clinical arena of medical school, in graduate medical education, and beyond.1 To make sound, evidence-based decisions, faculty involved in the admission process depend on empirical studies that examine the relationship of an applicant's academic performance before medical school to that individual's academic performance during medical school and afterwards. Studies have consistently shown that Medical College Admission Test (MCAT) scores and undergraduate grade-point averages (GPAs) are the most important indicators of students' future academic performances. Specifically, MCAT science scores and undergraduate science GPAs have been associated with preclinical academic performance.2 However, verbal scores on the MCAT and non-science GPAs have been more closely associated with performance in the clinical years, such as on the United States Medical Licensing Examination (USMLE) Steps 2 and 3.3 Correspondingly, the combination of GPAs and MCAT scores has been shown to be the best predictor of preclinical academic performance.4 The predictive strength of MCAT scores and GPAs is less clear when students' race and sex have been considered, and when performance has been followed longitudinally beyond the preclinical years. Men on average have outperformed women on the USMLE Step 1. The differences were moderated, but not eliminated, by statistical control for differences in prematriculation measures. Conversely, women have outperformed men on the National Board of Medical Examiners (NBME) Part II, though the differences were not as great as those observed between the scores of men and women on Part I, where men outperformed women.5 Control for differences in prematriculation measures and Part I performances increased the magnitude of differences between women and men on Part II. This phenomenon had been noted several decades earlier.6,7,8 Finally, the findings related to students' ages have been equivocal, often because age has been confounded with sex or undergraduate academic performance.9 Studies among racial groups have revealed substantial differences in performances on Part I. Although white students on average have scored highest, followed by Asian Americans, Hispanics, and African Americans, these gaps become narrower after controlling for MCAT scores and undergraduate GPAs.10 One might expect Asian Americans, who as a group have had the highest mean MCAT scores, to outperform other racial groups during medical school. However, two major studies across time and across medical schools have reported lower mean performance for Asian Americans than for white students in medical school.10,11 In summary, previous admission-prediction studies have looked at the predictive value of MCAT scores and GPAs for USMLE Step 1 performances among racial groups,10,12,13 clerkship performance during medical school,14 and a combination of Step 1 and clerkship performances.15 Other studies have ignored either students' age, race, or sex when examining the correlation between prematriculation measures and students' performances during medical school,16 or have studied characteristics such as race without controlling for GPAs and MCAT scores.11 We designed the present study to evaluate simultaneously the relative importances of MCAT scores, undergraduate GPAs, age, race, and sex in predicting performances on the three-step sequence of preclinical, clinical, and postgraduate licensing examinations. Method The sample consisted of 6,239 matriculants who entered Jefferson Medical College during the 30 years between 1968 and 1997, inclusive. The dependent variables were total scores on Parts I, II, and III of the licensing examinations of the NBME and total scores on Steps 1, 2, and 3 of the USMLE (the latter three examinations replaced the former three several years ago). Although scores on either of the preclinical examinations (Part I, Step 1) were available for every individual studied, scores on the second, clinical tests (Part II, Step 2) were available for only 5,887, because the others had either left medical school or not yet taken the test. Scores on the Part III and Step 3 examinations were collected prospectively for the 3,884 graduates (62%) who had given written permission and completed the examination at the time of the study. A separate multivariate linear regression model was generated for each of the six dependent variables. NBME scores were transformed from a mean and standard deviation of 500 and 100 to the USMLE scale of 200 and 20 to allow comparisons across the two time periods. Repeated scores were averaged. MCAT scores in earlier time periods were transformed to the current scale and repeated scores averaged using methods reported previously.17 Scores on science subtests were averaged to estimate an overall science score. Sex was coded 0 for men and 1 for women, who were 26% of the entire cohort. Students who were more than 23 years old at the time of matriculation (also 26% of the cohort) were coded 1 and others were coded 0. An earlier study of a portion of the cohort confirmed that this age cut-off serves as a proxy for nontraditional students.9 Racial—ethnic backgrounds, as defined by the Association of American Medical Colleges, consisted of the Asian, Oriental, or Pacific Islander groups (the Asian American group in this study); Hispanic (not white); black; and white. Students in each of the first three race categories were coded as either 1 for those in the group, or 0 for those not. The percentages for Asian American, Hispanic, and black were 8.2%, 1.4%, and 2.8%, respectively. The other students, who included 85.9% white and 1.7% in other racial groups with very small sample sizes, were not coded separately. Results Each of the six linear regression models shown in Table 1 was statistically significant (F test, p <.05). The proportions of variance explained ranged from a high for Step 1 of.26 to a low for Part III of.15. We report only the linear regression weights for the independent variables that were significant (t-test, p <.05). The b-coefficients for independent variables provide information about the absolute contribution of each variable as a predictor of the dependent variable. The beta-coefficients, which are transformations of the b-coefficients to a uniform scale across all independent variables, enable comparisons of the relative importance among the independent variables. For example, the b-coefficient of 4.26 for the MCAT science score in the USMLE Step 1 model indicates that a one-point increment in a student's MCAT science score raises his or her predicted Step 1 score by 4.26 points. Comparison of the beta-coefficient of.34 for the MCAT science score with the beta-coefficient of.21 for science GPA indicates that the unique contribution of the MCAT science score as a predictor of Step 1 is about one and a half times that of the science GPA. The validity of these interpretations of beta-coefficients assumes equivalent variability across independent variables, which has been documented in other published studies of portions of this cohort.18TABLE 1: B- and Beta-coefficients, Sample Sizes, and Proportions of Variance for Regressions Predicting Performances on NBME and USMLE Examinations from Applicant Data for Matriculants from 1968 through 1997*As would be expected, the contribution of the MCAT science score in predicting scores on the preclinical examination was more important than that of the science GPA. Being an older, nontraditional student at matriculation was unrelated to all scores after controlling for the other independent variables. The regression coefficients for women were negative for the NBME Part I, but insignificant for Step 1. However, being a woman was positively associated with the scores on USMLE Steps 2 and 3. Although being black was negatively associated with performances on Parts II and III, and being Hispanic negatively associated with performance on Part III, these patterns disappeared in the more recent USMLE examinations. Overall, the only consistent pattern related to age, race, or sex across all examinations was the negative regression weight for Asian American students. Discussion and Conclusion This longitudinal study examined the absolute and relative contributions of MCAT scores and undergraduate GPAs, along with age, race, and sex, in predicting students' performances on the sequence of three licensing examinations over the past three decades. The analysis reflected a large number of subjects, including a small fraction of students who reached Part I or Step 1 but did not graduate from medical school. Although the dependent variables were limited to licensing examinations, these are uniform measures that apply across all medical schools. As expected from many earlier studies, MCAT scores were consistently more valuable than were undergraduate GPAs as predictors of performance on licensing examinations, supporting their continued use in selection decisions.19 These relationships are stable across three decades and apply to the three examinations. Verbal scores tended to be better indicators of performances in the clinical and postgraduate tests. Although the non-science GPA never appeared in the six regression models, this may be due to the high correlation (r = 0.61) between science and non-science GPAs. There was no independent effect for older, nontraditional students after controlling for their undergraduate academic performances and MCAT scores. Earlier studies have indicated that, although underrepresented minorities have entered medical school with significant educational disadvantages and have continued to score lower than other students on some measures, their clinical performances were nearly equivalent to those of other students.20 In the present study, statistical control of the baseline differences at matriculation using regression analysis showed that underrepresented-minority students compared with white students performed less well than would have been predicted on the NBME in the earlier time period. However, this pattern disappeared in the recent time period. This change over time may have been due to the effectiveness of academic enrichment programs.21,22 It has been reported that the gap in MCAT scores and undergraduate GPAs between underrepresented minorities on one side and white and Asian American students on the other persists, supporting the need for programs aimed at enhancing students' academic preparation before medical school.23 The most striking finding is the large negative value of the b-coefficients as well as the beta-coefficients for Asian American students. This indicates that, after controlling statistically for science and verbal MCAT scores and undergraduate GPAs, these students performed less well compared with white students. Previous studies had revealed that Asian American students' performances during medical school were not as good as those of white students, without controlling for prematriculation measures.11 However, the differences between Asian American and the underrepresented-minority groups in Step 1 performances were significantly reduced after controlling for prematriculation measures.10 The findings of the present study indicate that Asian American students' performances fell below expectations on all NBME and USMLE examinations, after controlling for these prematriculation measures. One possible explanation for the decline in performance from the admission test to the licensing examinations may be that Asian American families are less able to influence academic achievement as their young adults mature. It has been reported that certain Asian American families place greater emphasis on doing well in school than do other groups.24 However, it is very important to consider that the sample used in the present study included a heterogeneous mix of Asian American students from families that left diverse cultures in Asia at different points in time. Further studies are needed to evaluate these subgroups, to investigate other measures of academic and clinical performance, and to better understand the factors that may influence Asian American students' performances in medical school and beyond.
Veloski et al. (Sun,) studied this question.