Answers are grounded in this paper and cite their sources.
Sign language conveys meaning through both manual gestures (hand shapes and movements) and non-manual cues such as facial expressions and body posture, yet previous studies have often modeled these channels separately, which can lose meaning. This study proposes a hybrid deep learning architecture that recognizes manual and non-manual gestures jointly. Each video is divided into 16-frame segments; YOLOv8 localizes the face, VGG16 and a recurrent neural network extract spatial and temporal facial features, and an improved 3D convolutional network (C3D) captures full-body spatiotemporal motion. The fused features are classified by an improved bidirectional long short-term memory network (iBiLSTM). The framework was evaluated on three benchmark datasets, PkSLMNM, RWTH-PHOENIX-Weather 2014T, and SILFA, using 5-fold cross-validation. On PkSLMNM, the model achieved a mean cross-validation accuracy of 95.79% (standard deviation 0.37%), outperforming LSTM (86.12%) and BiLSTM (89.15%) baselines, and it reached 93.84% and 94.21% accuracy on RWTH-PHOENIX-Weather 2014T and SILFA, respectively. An ablation study showed that fusing facial and body features outperformed either modality alone. These results indicate that combining manual and non-manual cues in a unified architecture improves both the accuracy and the cross-dataset robustness of dynamic sign language recognition.
No takes yet. Share an insight, caveat, or question.
Javaid et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: