Demonstrating Construct Validity of the American Board of Physical Medicine and Rehabilitation Part I Examination: An Analysis of Dimensionality

BACKGROUND: Ideally, high-stakes examinations assess 1 dimension of medical knowledge to produce precise estimates of a candidate's performance. It has not been reported whether the American Board of Physical Medicine and Rehabilitation Part 1 Certification Examination (ABPMR-CE-1) is unidimensional or not. OBJECTIVE: To examine the ABPMR-CE-1 to measure how many dimensions it assesses. DESIGN: Retrospective observational study. SETTING: We assessed examination results from the 2015 ABPMR-CE-1. PARTICIPANTS: A total of 489 deidentified candidates taking the 2015 ABPMR-CE-1. METHODS: A 1-parameter Item Response Theory (IRT) measurement model was utilized. A Principal Components Analysis (PCA) of standardized residual correlations was used to detect multidimensionality. MAIN OUTCOME MEASURE: Number of primary dimensions reflected in the 325 test questions. RESULTS: The results of the dimensionality analysis indicated that the ABPMR-CE-1 examination is highly unidimensional from a psychometric perspective. Expert content review of the substantive content of small contrasting clusters of questions provided additional assurance of the unidimensional nature of the examination. CONCLUSIONS: The ABPMR-CE-1 appears indeed to measure a single construct, which suggests a sound structure of the examination. It closely approximates the assumption of statistical unidimensionality. LEVEL OF EVIDENCE: Not applicable.