Research on speech conversion of Japanese dialects

Dialect is a broad conception containing both the variation of phonology and morphology. Converting a dialect speech into standard one could make it more understandable, while the reverse could make a speech full of regional characteristic. Currently, research on dialect conversion had been taken on many languages including Mandarin, English, and Arabic. But in the case of Japanese, there are a great number of dialect that lack enough speech data for training a modern sequence-to-sequence converter. In this research, a pipeline based on pretrained LM encoder and speech-LM was proposed to perform dialect speech conversion. Furthermore, by applying finetuning, non-parallel corpus in the form of either speech or text could be utilized for training the model, greatly enlarged the scale of accessible data. Finally, different structures were implemented and tested, pointed out future directions for the research.