Low Levels of Evidence on the Plastic Surgery In-Service Training Exam

BACKGROUND: The Plastic Surgery In-Service Training Exam is written by the American Society of Plastic Surgeons. Examinees reasonably infer that tested material reflects the Society's vision for the core curriculum in plastic surgery. The purpose of this study was to determine the levels of evidence on which credited answers to the examination questions are based. METHODS: Two recent Plastic Surgery In-Service Training Exams (2014 and 2015) were analyzed. Questions were categorized using a taxonomy model. Recommended journal article references for Level III (decision-making) questions were assigned a level of evidence. Exam sections were analyzed for differences in question taxonomy distribution and level of evidence. To look for studies with higher levels of evidence, a PubMed search was conducted for a random sample of 10 questions from each section. RESULTS: One hundred three Level I (25.8 percent), 138 Level II (34.5 percent), and 159 Level III (39.8 percent) questions were analyzed (p < 0.001). The hand and lower extremity section had the highest percentage of Level III questions (50.0 percent; p = 0.005). Journal articles had a mean level of evidence of 3.9 ± 0.7. The number of articles with a low level of evidence (IV and V) (p = 0.624) and the percentage of questions supported by articles with a high level of evidence (I and II) (p = 0.406) did not vary by section. The PubMed search revealed no instances of a higher level of evidence than the recommended reading list. CONCLUSIONS: A significant percentage of Plastic Surgery In-Service Training Exam questions test clinical management, but most are supported with a low level of evidence. Although that is consistent with low level of evidence of plastic surgery literature, educators should recognize the potential for biases of question writers.