Embodied AI as Coach: Integrating Facial Expression and Heart-Rate Biometrics into Adaptive Coaching Systems

Artificial Intelligence (AI) systems are increasingly used in human-centered domains such as coaching, education, and healthcare. However, most remain disembodied, relying solely on text or speech while neglecting the non-verbal cues essential to human communication. This paper advances the vision of embodied AI by proposing a multimodal framework that integrates facial expression analysis with biometric signals, heart rate, heart rate variability, and electrodermal activity, for real-time affect recognition. Grounded in embodied cognition, polyvagal theory, emotional intelligence frameworks, and affective computing, the study investigates how such integration can close the empathy gap in AI-mediated coaching. Specifically, it examines fusion strategies (early, late, and hybrid) for synchronizing heterogeneous signals and enabling adaptive coaching systems that dynamically adjust responses to users’ affective states. The expected contributions are both scientific, developing robust multimodal affective recognition, and applied, advancing empathetic, trustworthy, and personalized AI-driven coaching interventions.