STGAN-Q-Mix: Coordinated Spatiotemporal Learning for Polyglot Sign Language Interpretation

Multilingual sign language interpretation systems struggle with gradient instability, languagespecific overfitting, and inefficient coordination in decentralized multi-agent environments. Existing approaches fail to capture spatiotemporal dependencies while maintaining cross-lingual generalization across American Sign Language (ASL), Indian Sign Language (ISL), and Chinese Sign Language(CSL). We propose STGAN-QMix, integrating Spatiotemporal Gated Attention Networks with QMix reinforcement learning for polyglot sign interpretation. The architecture synergizes LSTM temporal modeling with Transformer self-attention for hierarchical spatiotemporal feature extraction, while QMix coordination enables optimal utility decomposition across distributed agents through value-based aggregation. This stabilizes gradient flow, eliminates redundant recurrent memory overhead, and enhances cross-lingual pattern recognition under real-time constraints. Our SignBot achieves 97.13% bidirectional communication by interpreting multilingual sign gestures into text and speech, while rendering speech back into sign animations. Experimental validation demonstrated superior prediction accuracy, gradient stability, and generalization capability across ASL, ISL, and CSL compared with state-of-the-art baselines. STGAN-QMix advances assistive communication technology by delivering a computationally efficient and scalable solution for seamless interactions between deaf-mute and hearing communities, achieving real-time inference latency under 127ms with competitive training overhead relative to comparable spatiotemporal architectures.