Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features

With the latest advancements, hand gesture recognition is becoming an effective way of communication and gaining popularity from a research point of view. Deaf people around the world need assistance, while sign language is only understood by a few people around the globe. It becomes challenging for untrained people to communicate easily, research community has tried to train systems with a variety of models to facilitate communication with deaf people and also human-computer interaction. Researchers have detected gestures with numerous recognition rates; however, the recognition rate still needs improvement. As the images captured via cameras possess multiple issues, the light intensity variation makes it a challenging task to extract gestures from such images, extra information in captured images, such as noise hinders the computation time, and complex backgrounds make the extraction of gestures difficult. A novel approach is proposed in this paper for character detection and recognition. The proposed system is divided into five steps for hand gesture recognition. Firstly, images are pre-processed to reduce noise and intensity is adjusted. The pre-processed images region of interest is detected via directional images. After hand extraction, landmarks are extracted via a convex hull. Each gesture is used to extract geometric features for the proposed hand gesture recognition (HGR) system. The extracted features helped in gesture detection and recognition via the Convolutional Neural Network (CNN) classifier. The proposed approach experimentation result demonstrated over the MNIST dataset achieved a gesture recognition rate of 93.2% and 90.2% with one-third and two-third training validation systems, respectively. Also, the proposed system performance is validated on the ASL dataset, giving accuracy of 91.6% and 88.14% with one-third and two-third training validation systems, respectively. The proposed system is also compared with other conventional systems. Different emerging domains such as human-computer interaction (HCI), human-robot interaction (HRI), and virtual reality (VR) are applicable to the proposed system to fill the communication gap.