Real-time Indian Sign Language word recognition through visual detection and temporal word construction

Indian Sign Language recognition is an important visual computing problem with directimplications for accessible human-computer interaction, yet practical deployment remainsconstrained by limited datasets, signer variability, video-quality differences and the diffi-culty of converting frame-level gesture detections into coherent words. This paper presentsan end-to-end framework for real-time Indian Sign Language word recognition from video.The proposed system integrates multi-source dataset construction, size-aware data augmen-tation, lightweight visual detection, video frame processing, parameter optimisation and aword-building strategy designed to handle repeated letters and temporal inconsistencies. Adataset of 50,000 images covering 36 classes, including digits 0–9 and letters A–Z, is createdby combining a public Indian Sign Language dataset with custom recordings from multipleusers. To improve robustness, the study applies tiered augmentation, with moderate trans-formations for larger source data and stronger transformations for smaller custom subsets.The recognition model is trained using a real-time object-detection architecture and evalu-ated in terms of precision, recall and mean average precision. The reported results show highdetection performance, with final precision of 99.74%, recall of 98.43% and mAP@50–95 of99.40% on the validation data. The system further incorporates video-quality enhancementand interactive parameter tuning to support cross-user testing, achieving real-time opera-tion at approximately 15–20 frames per second with low latency. The study demonstratesthe feasibility of combining visual detection and temporal post-processing for Indian SignLanguage spelling recognition, while also highlighting the need for stronger external valida-tion, reproducible benchmarking and more systematic comparison with recent sign-languagerecognition methods