Real-Time Sign Language Recognition System for Enhancing Emergency Response Among Hearing-Impaired Patients

This study presents the design and implementation of an Image-Based Sign Language Recognition System aimed at facilitating emergency communication with hearing-impaired patients. The system integrates deep learning and computer vision techniques to recognize real-time emergency-specific hand gestures such as Accident, Pain, Cold, Fever, and Help. A custom dataset comprising 250 manually captured images was developed to train and evaluate the model, with 50 images per class to ensure balanced representation. The hybrid architecture employs YOLOv8 for gesture detection and a Convolutional Neural Network (CNN) for classification. YOLOv8 efficiently localizes hand regions within video frames, while the CNN interprets the extracted gestures into corresponding emergency labels. The system was implemented using Python, OpenCV, and TensorFlow, and tested under various lighting conditions, hand orientations, and background environments to simulate real-world scenarios. Experimental results demonstrate that the integrated YOLOv8-CNN framework achieved a training accuracy of 99% and a testing accuracy of 93.5%, with high recall (0.927), F1-score (0.933), and precision (0.933) across all gesture classes. Performance analysis shows that YOLOv8 attained a mean average precision (mAP@50 = 0.984, mAP@50-95 = 0.915), indicating excellent detection capability, while the CNN maintained consistent classification accuracy across categories. The results confirm the system’s robustness, reliability, and real-time responsiveness, demonstrating its potential to enhance emergency communication efficiency for hearing-impaired individuals in healthcare settings.