Wearable Saudi Sign Language Recognition Device Based on Neural Network

In Saudi Arabia, over 750,000 individuals are deaf or mute, often facing communication barriers due to limited sign language familiarity among the general population. This research introduces a wearable device leveraging neural networks to recog-nize Saudi Sign Language, aiming to bridge this communication gap. The device simultaneously processes data from both hands, integrating inputs from the non-dominant hand with those from the dominant. Three primary phases define the system: data acquisition, machine learning model development, and real-time sign recognition. Data were gathered using electromyography (EMG) and inertial measurement unit (IMU) sensors, facilitated by the Arduino IDE, with two Arduino Nano 33 BLE micro-controllers managing collection and transmission. The dataset, comprising 25 signs from four subjects with each sign repeated 20 times, informed the machine learning process. Utilizing Python, a model incorporating dual Convolutional Neural Networks for feature extraction from sensor readings and dense neural networks for training was built. Subsequent sign recognition was realized by deploying the model on Arduino via TensorFlow Lite. Impressively, this innovative system boasts a 92 % accuracy rate, marking a significant stride toward inclusive communication solutions.