Answers are grounded in this paper and cite their sources.
Low proficiency in fundamental motor skills among children highlights the need for valid, feasible, and reliable assessment methods to support instruction and population-level monitoring. This study examined agreement between artificial intelligence (AI)-generated forward roll scores and expert consensus ratings and whether criterion-level AI support was associated with closer agreement between human ratings and expert consensus, higher inter-rater reliability, and lower perceived workload within the Fundamental Motor Skills in Sport (FUS) test. Scores from an AI model were compared with expert consensus ratings in 66 forward roll videos spanning the full 0–5 total-score range using Cohen’s kappa and quadratic weighted kappa. In a counterbalanced experiment, 20 raters from Malta and Poland assessed two independent sets of 22 videos, one with and one without criterion-level AI support. Human rating error and agreement were evaluated against a separate blinded expert consensus, while inter-rater reliability was estimated descriptively. Perceived workload was assessed after each condition using the NASA Task Load Index. AI-generated criterion-level scores showed substantial to almost perfect agreement with expert consensus (kappa 0.73–0.85). Quadratic weighted kappa between AI-generated and expert-consensus total scores was 0.95 (95% CI 0.92–0.97). Human ratings with AI support more closely matched the separate blinded expert consensus: mean absolute error was 0.68 (95% CI 0.53–0.85) versus 0.99 (95% CI 0.85–1.13), and quadratic weighted kappa was 0.79 (95% CI 0.65–0.87) versus 0.60 (95% CI 0.44–0.70). Error was lower for 19 of 20 raters (Wilcoxon p < 0.001, conditional on sampled videos). Descriptive ICC(2,1) point estimates were higher with AI support in all four country-by-video-set comparisons. After Holm correction, only perceived effort was significantly lower with AI support ( p = 0.045). Under standardized video conditions, the AI system showed strong agreement with expert consensus. Human ratings made with AI support also more closely matched a separate blinded consensus, and raters reported less effort. These task-specific findings support further evaluation of criterion-level AI recommendations as a decision aid for human-supervised forward roll scoring. The rater findings concern the participating master’s students. Extension to practicing teachers, other age groups, other skills, or routine school settings requires separate validation.
No takes yet. Share an insight, caveat, or question.
Makaruk et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: