A two-stage sign language recognition method focusing on the semantic features of label text

The ability to recognize sign language is an indispensable technology that plays a crucial role in facilitating communication between individuals who are deaf or hard of hearing. It is of utmost importance to comprehensively understand the nonverbal expressions employed by the hearing impaired. In order to enhance the efficacy of sign language recognition technology, it is imperative to focus on language modeling and improve the utilization of linguistic elements. At present, much attention in sign language recognition techniques that integrate language modeling is directed toward the translation of GLOSS to text in research related to Sign Language Translation (SLT). Our paper, however, proposes a creative approach that involves the linguistic modeling of the corresponding text of sign language during the process of converting signs to GLOSS. Specifically, we have implemented a text correction module that uses a front-mounted sign language recognition module to make preliminary predictions. The corrected GLOSS sequence is then used to obtain the final recognition result with higher accuracy. Our framework was tested on the RWTHPHOENIX-Weather-2014-T dataset and CSL dataset to evaluate its effectiveness in recognizing sign language on a large scale. The experimental results demonstrate that the proposed method significantly enhances the accuracy of the sign language recognition model.