Answers are grounded in this paper and cite their sources.
Continuous sign language translation (CSLT) aims to convert continuous signing into spoken-language text, but fixed multimodal fusion may fail to adapt to changing visual cues.We propose a graph-enhanced multimodal Transformer using gloss-assisted training with gloss-free inference.The framework combines RGB appearance, skeletal pose, and facial landmarks through adaptive cross-modal weighting and a temporal saliency gate.A CTCbased recognition objective and an autoregressive translation objective are jointly optimized during training, while gloss prediction is not required during inference.On PHOENIX-2014T, the model achieved 17.80 ± 0.20% WER, 25.40 ± 0.12 BLEU-4, and 34.80 ± 0.15 ROUGE-L.On CSL-Daily, it improved BLEU-4 by 0.70 over the strongest baseline.The framework achieved 22 FPS inference throughput under the reported configuration, while ablation and robustness experiments indicated contributions from multimodal fusion and resilience to degraded inputs.
No takes yet. Share an insight, caveat, or question.
S et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: