This systematic survey comprehensively examines the integration of multimodal affective computing (MAC) with virtual rehabilitation (VRe) and proposes an Affective-driven Virtual Rehabilitation (AdVRe) framework. AdVRe dynamically adapts rehabilitation content by real-time monitoring of patients' emotional states via physiological, behavioral, and interaction signals. The review addresses key research questions, including emotion model selection, signal acquisition methods, multimodal fusion algorithms, applications, and datasets. Technical foundations emphasize MAC's superiority over unimodal approaches in accuracy and robustness, enabling closed-loop 'emotion-in-the-loop' adjustments to optimize engagement and outcomes. Applications span limb motor training, PTSD/anxiety therapy, ASD social cognition, and chronic pain management across non-, semi-, and immersive VR platforms. Challenges include (i) emotion recognition instability under clinical variability, (ii) scarce rehabilitation-specific datasets, (iii) computational constraints for real-time processing, and (iv) data privacy concerns. Opportunities lie in contactless sensing, edge computing, federated learning, and personalized emotional modeling. The study concludes that MAC-enhanced VRe significantly improves rehabilitation personalization and efficacy, though interdisciplinary efforts are needed to advance clinical translation by optimizing algorithms, making sensors accessible, and establishing standardized ethical frameworks.
Zhao et al. (Thu,) studied this question.