Refinement of a formative assessment tool for an informed consent (EPA11) curriculum

Purpose Previous work found an informed consent curriculum and corresponding novel formative assessment tool for graduating fourth-year medical students preparing for a surgical-based residency to be effective based on student-performance and end-of-course-survey data. However, students underperformed on certain skills indicating the opportunity for refinement of the assessment tool. This study aims to identify and refine areas of the assessment tool used in the informed consent curriculum. Methods A total of 325 medical students across two-academic years (22–23, 23–24) received an informed consent curriculum. Students practiced a specialty-specific informed consent encounter with a standardized-patient (SP). Student performance was rated by the SP using a 22-line-item communication skills tool, using a defined rubric (scale 0, 1, 2) for each skill. SP-rated student performance data were tallied for each line-item. Student performance data was analyzed using an exploratory factor analysis (principal axis factoring with Promax rotation) and internal reliability test with IBM SPSS Version 30 statistical package. Results Exploratory factor analysis was conducted on 22 communication skills. Sampling adequacy was acceptable (KMO = 0.66; Bartlett’s χ 2 (231) = 968.27, p < .001). The scree plot indicated retention of approximately three factors, and the initial eigenvalue criterion yielded eight factors explaining 34.8% of variance. Using a three-factor model, retained factors represented eight skills defined within dimensions of patient engagement, diagnosis education, and procedure explanation. Items with low loadings (loading < 0.40) were excluded. The three factors accounted for 20.91% of the variance. The Cronbach’s alpha for the 22-line-item skills assessment was acceptable (alpha = 0.660). Conclusions Exploratory factor analysis revealed three main constructs (patient-centered engagement, diagnosis education, and procedure explanation) of the informed consent formative assessment tool, indicating targetable areas for feedback to students.