Validating Automated Feedback in an AI-Based Interactive Learning Environment for Dance: Agreement Between Expert Choreographers and the ReAL-T

The increasing use of automated feedback in education highlights the need for rigorous validation of AI-based assessment tools, particularly in performance-based domains such as dance. This study examined the validity of the Real-time Augmented Feedback in Learning Tool (ReAL-T), an AI-based scoring engine designed to support screen-based learning and assessment for beginner dancers. Twelve adult beginners completed a learning protocol involving four choreographies, generating 96 recall performances. Three expert choreographers independently rated each performance on a 1–5 scale for body line and form and precision, while ReAL-T generated parallel scores using the same criteria. Inter-rater reliability among choreographers was evaluated using intraclass correlation coefficients (ICCs) and Kendall’s coefficient of concordance (W). Agreement between expert ratings and ReAL-T was examined using ICCs and Kendall’s W on a combined overall score. Expert ratings demonstrated excellent agreement across both categories (single-rater ICCs ≥ 0.85; Kendall’s W ≥ 0.87). When ReAL-T was included as an additional rater, agreement remained high (single-measure ICC = 0.84; Kendall’s W = 0.89). These findings indicate that the ReAL-T minimum viable product produces automated scores that closely align with expert judgement, supporting its use as a validated automated feedback component for beginner dance education.