Characterizing how surgical residents evaluate faculty: a document analysis of faculty teaching evaluation tools

Abstract Background Feedback on faculty teaching is essential for faculty development and education quality. Often in surgical training programs, this feedback is provided through resident evaluations of faculty. The content and quality of faculty teaching evaluations being used are, however, unknown as no standardized tool exists. The aim of this study was to identify the characteristics and evaluation domains of faculty teaching evaluation tools used in surgical residency programs. Methods Using convenience sampling, blank faculty teaching evaluation forms completed by residents were collected from university-based surgery residency programs across multiple U.S. geographic regions, including large public and private academic medical centers. A document analysis was performed using a mixed-methods approach to conduct a quantitative analysis to characterize the tools’ design features using descriptive statistics and perform a qualitative analysis of all question items using inductive content analysis to determine the evaluation domains. Results Fourteen faculty teaching evaluation forms from thirteen U.S. surgery departments met the sampling criteria and were included in the final analysis. Quantitative analysis showed that the mean number of questions per evaluation was 14.7 (median 13; range 4–23), composed of rating questions (median 10.5; range 0–20), free text comments (median 3; range 1–12), and checklist questions (median 0.5; range 0–2). The analysis also identified five styles of response anchors to each question including sequential, agreement, frequency, binary, and open-ended, and three performance levels for assessed faculty traits or behaviors including below expected performance, at expected performance, and above expected performance. Qualitative analysis of 206 question items revealed four major evaluation domains including teaching effectiveness (53.8% of codes), professionalism (25.5%), clinical performance (11.6%), and overall performance (9.1%) from the faculty teaching evaluation tools. Conclusions Surgical faculty teaching evaluations vary widely in design and frequently conflate teaching with professionalism and clinical performance. This domain mixing poses challenges for score interpretation and threatens construct validity if evaluations are used to assess teaching effectiveness exclusively. The findings suggest a consensus-based approach to defining core teaching domains and evidence-informed design principles, while allowing flexibility for local implementation. This study demonstrates the value of document analysis for critically examining assessment tools and provides empirical groundwork to inform future faculty evaluation efforts in surgical education.