Appraising and applying evidence about a diagnostic test during a performance-based assessment

BACKGROUND: The practice of Evidence-based Medicine requires that clinicians assess the validity of published research and then apply the results to patient care. We wanted to assess whether our soon-to-graduate medical students could appraise and apply research about a diagnostic test within a clinical context and to compare our students with peers trained at other institutions. METHODS: 4th year medical students who previously had demonstrated competency at probability revision and just starting first-year Internal Medicine residents were used for this research. Following an encounter with a simulated patient, subjects critically appraised a paper about an applicable diagnostic test and revised the patient's pretest probability given the test result. RESULTS: The medical students and residents demonstrated similar skills at critical appraisal, correctly answering 4.7 and 4.9, respectively, of 6 questions (p = 0.67). Only one out of 28 (3%) medical students and none of the 15 residents were able to correctly complete the probability revision task (p = 1.00). CONCLUSIONS: This study found that most students completing medical school are able to appraise an article about a diagnostic test but few are able to apply the information from the article to a patient. These findings raise questions about the clinical usefulness of the EBM skills possessed by graduating medical students within the area of diagnostic testing.