Background Teleconferencing has become essential for professional collaboration, yet many platforms inadequately support deaf, hard-of-hearing, and multilingual participants. Over 430 million people worldwide require rehabilitation for disabling hearing loss, and this figure is projected to exceed 700 million by 2050. Traditional accessibility solutions face substantial limitations including high costs, limited availability, and scheduling constraints. Extended Reality (XR) technologies offer new opportunities to create immersive, inclusive communication environments Methods INTERACT (Inclusive Networking for Translation and Embodied Real-Time Augmented Communication Tool), is an AI-powered XR platform that integrates real-time speech-to-text transcription, International Sign Language (ISL) translation via 3D avatars, multilingual support, and sentiment analysis within immersive virtual environments. Built on the CORTEX21 architecture and leveraging Meta Quest 3 headsets, the platform employs state-of-the-art AI models including Whisper for speech recognition, NLLB for multilingual translation, RoBERTa for sentiment analysis, and Google MediaPipe for gesture extraction. Pilot demonstrations were conducted in two phases: One demo involving technical experts from the academic/industry sector and another demo involving directly deaf community stakeholders. All participants provided written informed consent prior to participation, in accordance with the ethical protocols approved by DASKALOS-APPS’s Ethics Review Board. Results Pilot demonstrations achieved 92% user satisfaction, exceeding 85% transcription accuracy, and 90% sentiment detection precision. Participants rated their overall experience at a mean of 4.6 out of 5.0, with 90% expressing willingness to participate in future testing. The system successfully supports up to 1,000 concurrent users with minimal latency degradation. Load testing validated system performance under stress conditions with 10,000 simultaneous requests achieving 900–1,000 requests per second with zero failures. Conclusions This work represents the first XR-based teleconferencing platform combining real-time ISL translation via 3D avatars with multilingual speech transcription and sentiment-aware feedback in a cohesive, immersive solution. The results demonstrate significant potential for transforming accessibility in educational, cultural, and professional settings.
Tantaroudas et al. (Tue,) studied this question.