Speech synthesis with diverse laughter types using artificial data

Nonverbal speech synthesis that conveys not only textual content but also expressive information is important for building human-like, natural dialogue systems. In particular, laughter plays a crucial role in expressing humor and emotion. While prior work has enabled the synthesis of laughs or speech-laughs, generating a wide variety of laughter types, such as giggles, loud laughter, and tearful laughter, remains a significant challenge. This difficulty stems mainly from the scarcity of training data and the highly individual nature of laughter, which makes it hard to model and synthesize high-quality speech. In this study, we aim to synthesize speech that includes diverse types of laughter of a target speaker. For this purpose, we propose a training method that improves both the quality and variety of laughter synthesis. Specifically, we first generate synthetic speech samples that include various types of laughter using a large-scale pretrained TTS model. We then conduct pretraining on this synthetic data, followed by speaker-specific fine-tuning using a small amount of real laughing speech data. This approach enables the generation of a wide range of expressive laughter from the target speaker, even when only a small number of samples are available. We report our methodology in detail and present experimental results.