RL-Robotutor: a hierarchical reinforcement learning framework for multimodal robot tutoring

Personalised robot tutoring requires real-time pedagogical decision-making under uncertain and multimodal learner states. Existing intelligent tutoring and robot-assisted learning systems often rely on static rules, single-modality learner modelling, or short-term correctness-oriented optimisation, which limits their capacity to balance knowledge acquisition, engagement, retention, and safe embodied interaction. RL-Robotutor is presented as a hierarchical reinforcement learning framework that integrates cognitive knowledge-tracing signals, facial-expression and gaze-derived affective cues, and behavioural interaction logs through transformer-based fusion. A PPO-trained meta-policy selects among specialised modules for learner assessment, curriculum adaptation, strategy selection, and reflective feedback, while a hybrid reward combines immediate learning performance, engagement, long-term learning gain, and safety. The evidence is reported by source: five public knowledge-tracing datasets assess the cognitive prediction component; controlled simulator experiments assess policy learning, reward ablation, and multimodal fusion; and a randomised robot-tutoring study assesses embodied interaction with 120 undergraduate participants. RL-Robotutor outperforms rule-based, flat reinforcement-learning, knowledge-tracing, and LLM-based comparators. In the human study, the learning gain is 0.62, compared with 0.40 for the rule-based robot condition. The full hybrid reward improves simulator learning gain by 19% over correctness-only optimisation, and full multimodal fusion improves simulator engagement by 17% over text-only modelling. Jetson Orin deployment achieves 48 ms inference latency with an absolute AUC reduction of 0.007, supporting real-time edge execution.