A virtual teaching assistant system for dance teaching combining deep learning and reinforcement learning

Dance instruction often suffers from the traditional one-on-one training approach for reasons of adaptive learning, feedback and understanding of movement. In digital dance learning methods, there is no individual feedback and skill development. In this work, the authors design a virtual teaching assistant system entitled Virtual Intelligent Reinforcement Teaching Unit for Optimized Skill-oriented Output (VIRTUOSO) using deep learning and reinforcement learning. This system has been developed to enhance the teaching of dance. The objective is to develop an intelligent system that evaluates student performance, provides immediate feedback, and adjusts instructional strategies based on each student’s ability level. Motion recognition is done with a 3D Spatio-Temporal Graph Convolutional Network (ST-GCN) and movement sequence evaluation with a BiLSTM-based quality estimator. Using Reinforcement Learning (RL), a reinforcement learning agent adapts its instruction tactics to account for learner failures and feedback loops. VIRTUOSO decreases teacher intervention time by 42% and increases student performance accuracy by 28% compared to static tutorials. User engagement and learning retention are greatly improved by interactive and adaptive feedback. The model was trained and validated using capture records of participants practicing Ballet, Hip-Hop, and Salsa during Preparation, Execution, and Recovery dance postures from the Dance Simulation System Dataset. The proposed algorithm consistently scores 92.5% with competent teachers and adapts to over 15 different dance genres. The RL component also enables the assistant to learn and refine its feedback tactics. VIRTUOSO has excellent potential as a scalable, intelligent, individualized dance-teaching system.