Conversion of Images to Indian Sign Language Using Object Detection

The present paper introduces a web-based application that automatically converts images to Indian Sign Language (ISL) signs to provide accessibility for the hearing-impaired. The system includes a custom-trained object detection with YOLOv8, ISL API for fetching SiGML data and JASigning for animating the ISL signs using "Marc" avatar. Users can upload images or select from predefined samples and the system will show the detected objects with their respective ISL signs. The system achieved an average precision of 92% for object detection and a 95% success rate for fetching sign data over a test dataset of 100 images of varied categories. Due to the user-friendly environment and high accuracy, the system is a viable educational tool and for daily usage, to fill the communication gap between the hearing-impaired. The work is to expand object varieties and information coverage of sign language in the near future.