An Intelligent Computer-Based System for Sign Language Tutoring

ABSTRACT A computer-based system for sign language tutoring has been developed using a low-cost data glove and a software application that processes the movement signals for signs in real-time and uses Pattern Matching techniques to decide if a trainee has closely replicated a teacher's recorded movements. The data glove provides 17 movement signals from Bend sensors and push down switches and is used initially by the teacher to record selected signs. A trainee can study an animated 3D model of a hand and arm showing the recorded sign movements at leisure, before using the data glove to try and replicate the sign. Four Arabic Sign Language (ArSL) teachers provided 65 common posture and gesture signs, and then 10 trainees from the local community were asked to evaluate the system. They achieved an overall average accuracy of 93.8% in replicating the signs, despite finding the gestures harder than the postures to perform, and found approximately 18% of the signs difficult because of particular thumb/finger and wrist bending angles. Both the teachers and the trainees were familiar with these signs, and a usability questionnaire revealed that they preferred this approach to sign-language tutoring than the traditional human based method they had already experienced.