Multi-rater assessment of entrustable professional activities among novice nurses in a standardised training programme: A cross-sectional study

Entrustable professional activities (EPAs) connect observable clinical work with decisions about the supervision a learner requires. Evidence remains limited on how EPA ratings assigned to novice nurses differ among assessment roles and how closely those ratings agree. This cross-sectional study analysed 215 complete triads comprising a novice nurse, clinical mentor and nurse manager at a tertiary teaching hospital in China. Twelve EPAs were rated using a five-level ordinal entrustment scale. Role-related differences were examined using Friedman tests followed, when indicated, by paired Wilcoxon signed-rank tests. Agreement was evaluated separately using linear-weighted kappa, exact agreement and within-one-level agreement. Independent training-year cohorts and clinical specialties were compared using Kruskal–Wallis tests. Holm adjustment was applied to groups of related tests, while Bonferroni adjustment was used for the three post-hoc role comparisons within each EPA. Rating distributions differed among roles for 10 of 12 EPAs after Holm adjustment, although Kendall’s W values were small. Linear-weighted kappa ranged from − 0.036 to 0.316; exact agreement ranged from 39.1% to 70.2%, while within-one-level agreement ranged from 87.4% to 100.0%. All 36 EPA-by-role omnibus comparisons across training-year cohorts were significant after Holm adjustment, whereas a specialty difference remained only for nurse-manager ratings of EPA7. No adverse-event comparison remained significant after multiplicity adjustment. The three assessment roles provided distinct perspectives, with generally low chance-corrected agreement. The findings support cautious interpretation of multisource ratings, explicit rater calibration and prospective evaluation of stage-specific benchmarks. Longitudinal studies are needed to examine individual development and determine whether EPA ratings predict subsequent adverse events.