Speech recognition in videos using features of different enhancement filters

Video Speech Recognition (VSR) is the ability of software to extract spoken text from input sources, such as offline videos and YouTube. It achieves a good recognition accuracy rate for videos with clean environments. However, with unwanted noisy elements in speech, this presents a major challenge. Depending on where the noise originates from, there are considerable differences in the impact of various noise kinds on video speech recognition, including background music, multiple speakers, microphone quality, and dialects. It is not possible to create a filter for every kind of noise because the sources of noise might differ greatly and the speech environment’s circumstances can change over time. Hence, instead of using a single filter for each type of noise, this study proposed to combine features from a small number of noise removal filters applied to the same video speech signal, resulting in a better VSR output. According to experimental results, using the proposed framework increases the accuracy of video speech recognition when evaluated on YouTube videos with different noise levels. In test experiments, the proposed framework was evaluated against state-of-the-art approaches. The accuracy mean of the relevant current approaches was improved by 16.67% and against the best accuracy of them by 7.78%. This research contributes to the video speech recognition topic as the proposed framework can facilitate the extraction of spoken phrases in millions of hours of video available online, indexing them, and then searching for them through their index.