People with disabilities mostly communicate using sign language, but the public still has little understanding of the Indonesian Sign Language System (ISLS). This causes obstacles in daily interactions. Advances in artificial intelligence technology, especially artificial neural networks, open opportunities in sign language recognition, but are still in the development stage. This study aims to build a ISLS sign language recognition model using the LSTM approach and MediaPipe Hands. The method of collecting hand keypoint data, 25 sequences per gesture, and 36 alphabetic and numeric gestures. The dataset is divided into three categories, namely 80% training, 10% validation, and 10% testing. The model developed to handle sequential data from hand gestures using the LSTM architecture. The results of the study can be shown model accuracy of 97.1%, average macro precision of 97%, recall of 96.6%, and F1-score of 96.4% and weighted average precision of 97.4%, recall of 97.1%, and F1-score of 97%. The results show that the combination of LSTM and MediaPipe can detect ISLS gestures with high accuracy. This can be used as a potential solution in automatic sign language translation, so that this model can improve the inclusiveness of communication for people with disabilities. Further research can be developed using a more accurate hand recognition framework, as well as improving data pre-processing, and exploring deep learning (DL) methods such as SSD, YOLO, or Faster-RCNN. In addition, pose and facial recognition can be added to improve accuracy in gesture recognition more comprehensively.
Utomo et al. (Tue,) studied this question.