What question did this study set out to answer?

The aim is to develop a real-time system that translates sign language into text and speech using AI.

April 25, 2026Open Access

Real-Time Sign Language to Text/Speech Translator

Key Points

The aim is to develop a real-time system that translates sign language into text and speech using AI.
Utilized Google MediaPipe for hand landmark detection
Developed a custom CNN for gesture classification
Created a dataset of 15 commonly used signs from 5 users, yielding 15,000 samples.
CNN achieved 96% training accuracy and 91% validation accuracy
Average response time was 250ms
Usability testing yielded an SUS score of 84.2, indicating excellent user satisfaction.

Abstract

This paper presents a Real-Time Sign Language Recognition and Translation System that converts hand gestures into text and speech using AI and computer vision. The system uses Google MediaPipe for 21-point hand landmark detection, a custom Convolutional Neural Network (CNN) for gesture classification, and Text-to-Speech (TTS) synthesis — requiring only a standard webcam, with no specialized hardware needed. A custom dataset of 15 commonly used signs was collected from 5 users, yielding 15,000 samples. The CNN achieved 96% training accuracy and 91% validation accuracy with an average response time of 250ms. The system was deployed as a web application (Next.js + TensorFlow.js), desktop app (Python), and REST API (FastAPI). Usability testing with 12 participants produced a System Usability Scale (SUS) score of 84.2 (Excellent). The system targets real-world use in education, healthcare, and public services, promoting accessibility and inclusion for the hearing-impaired community. Developed as a B.Tech (CSE) final year project at Shri Ramswaroop Memorial University, Lucknow

Read Full Paperexternally

Perguntar à IA

Bookmark

View Full Paper