Open-Source Full-Duplex Conversational Datasets for Natural and Interactive Speech Synthesis

Full-duplex, spontaneous conversational data are essential for enhancing the naturalness and interactivity of synthesized speech in conversational TTS systems. We present two open-source dual-track conversational speech datasets, one in Chinese and one in English, designed to enhance the naturalness of synthesized speech by providing more realistic conversational data. The two datasets contain a total of 15 hours of natural, spontaneous conversations recorded in isolated rooms, which produces separate high-quality audio tracks for each speaker. The conversations cover diverse daily topics and domains, capturing realistic interaction patterns including frequent overlaps, backchannel responses, laughter, and other non-verbal vocalizations. We introduce the data collection procedure, transcription and annotation methods. We demonstrate the utility of these corpora by fine-tuning a baseline TTS model with the proposed datasets. The fine-tuned TTS model achieves higher subjective and objective evaluation metrics compared to the baseline, indicating improved naturalness and conversational realism in synthetic speech. All data①②, annotations, and supporting code for fine-tuning and evaluation are made available to facilitate further research in conversational speech synthesis. <fn id="fn1" specific-use="footnote"> ¹ <uri xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://magichub.com/datasets/multi-stream-spontaneous-conversation-training-datasets_chinese/">https://magichub.com/datasets/multi-stream-spontaneous-conversation-training-datasets_chinese/</uri>. </fn> <fn id="fn2" specific-use="footnote"> ² <uri xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://magichub.com/datasets/multi-stream-spontaneous-conversation-training-datasets_english/">https://magichub.com/datasets/multi-stream-spontaneous-conversation-training-datasets_english/</uri>. </fn>