Automating Multilingual Video Dubbing Using Deep Learning and Audio Synthesis

The paper offers a holistic approach to video translation into several languages that combines deep learning, and audio synthesis techniques, in order to automate the process. Exploiting recent advance in automated speech translation and multimodal translation, we shall develop an approach reproducing high-quality dubbed video materials efficiently and at a large scale. The methodology includes: 1. extraction of audio from the original videos, 2. transcription and translation of audios, 3. synthesis of translated audio tracks, and 4. fusion them with the audio of original videos in order to get final multilingual dubbed videos. Trial findings provide evidence that the suggested approach is able to generate high-quality and indistinguishable from original counterpart dubbed videos among different languages and video formats.