Answers are grounded in this paper and cite their sources.
This research improves automatic speech recognition via an expanded dataset, suggesting increased accuracy from using LRS2 and LRW resources.
Automatic speech recognition by articulation (lip reading) remains a relevant problem, despite the existing solutions. One of the key difficulties is the insufficient and imbalanced training data, which reduces the accuracy, and most importantly, the generalizing ability of the models. Purpose: to increase the generalizing ability of the automatic speech recognition system by articulation by expanding the database and fine-tuning the model. Methods: expanding the LRW dataset, containing videos of individual words, by processing and adapting the LRS2 dataset, containing video of phrases. The neural network architecture used is a combination of modified ResNet-18 and BiGRU. Results: expanding the dataset with visually similar samples from LRS2 dataset reduced the class imbalance and increased the amount of training data. Fine-tuning the model on a mixed dataset increased its accuracy and generalizing ability compared to the baseline, bringing its application closer to real-world conditions. Practical relevance: the developed method can be applied to improve automatic speech recognition systems in data-limited scenarios, namely, the customized LRS2 dataset can be used similarly and in combination with the LRW dataset. Discussion: a promising direction for future research is further dataset expansion using additional sources, as well as the development of new neural network architectures for efficient lip-reading.
No takes yet. Share an insight, caveat, or question.
V. O. Yachnaya (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: