Reflective ability in medical trainees: reliability and practicality of a measurement instrument and assessment in a large cohort

Purpose Characterization and measurement of reflective capacity (RC) - the ability to learn from experience to inform future learning and practice—in hospital trainees are critical to the design of medical education programs; testing interventions requires effective assessment. This study assessed RC among anesthesia trainees using the Reflection Evaluation for Learners’ Enhanced Competencies Tool (REFLECT), which evaluates reflective writing. The study explored variation by training level, age, program, gender, and examined reliability and validity against an independent reflection measure. Method A prospective observational study (February–May 2021) included anesthesia trainees across four training levels, four programs, and varied prior RC experience and intended sub-specialties. Participants completed the REFLECT instrument (using simulated and personal events) and the Residency Level Cognitive Behavior Survey (rCBS). Potential influencing factors (scenario realism, perceived importance, prior reflective writing) were examined. Data were analyzed for internal consistency (t-tests, ANOVA, Pearson’s r) and interrater reliability (Cronbach’s α, intraclass correlation coefficient). Results Participants (n = 172) demonstrated moderate RC (mean = 2.5/4, SD = 0.45), with no significant differences by training level, program, gender, or age. Greater perceived realism and prior reflective writing were independently associated with higher RC. Those with prior RC training rated reflection as more important and useful. Personal and simulated event scores were similar. REFLECT showed excellent interrater reliability (Cronbach’s α = 0.95) and moderate correlation with the rCBS Reflection domain (r = 0.31, P = .001). Conclusions REFLECT demonstrates robust psychometric properties for assessing RC in anesthesia trainees. Moderate RC scores support the need for interventions to enhance reflection. Although limited to one specialty and not assessing application to practice, the instrument’s general structure suggests potential utility across postgraduate medical training contexts.