Real-time speech enhancement using temporal envelope modulation and hybrid neural network algorithms for improved speech-to-text conversion

The algorithm converts improved speech into text for speech synthesis applications. This approach combines hybrid neural networks and offers superior quality and intelligibility while minimising word error rates by incorporating temporal envelope modulation analysis at the speech prosody level. The algorithm works with enhanced speech with adaptive gain control having 16-channel BPFs, envelope detectors, and LPFs and combining CNNs and LSTMNs to reduce background noise and enhance accuracy. Then, the output will be text using HMM on the enhanced speech. Various types of noise, such as babble SSN, were tested through experimental evaluations on real-time recorded samples using PRAAT software and TIMIT and Noizeus noise databases at various SNR levels. The algorithm showed a drastic performance improvement, evaluated through PESQ, STOI, and cosine similarity scores, thus showing a gain of 39% in PESQ and 33.3% in STOI. In addition, compared with noisy speech, up to 3% reduction in word error rate was found, along with improved recognition accuracy of spoken words by up to 90% for phrases of 8-10.