Speaker-conditioned phrase break prediction for text-to-speech with phoneme-level pre-trained language model

This paper advances phrase break prediction (also known as phrasing) in multi-speaker text-to-speech (TTS) systems. We integrate speaker-specific features by leveraging speaker embeddings to enhance the performance of the phrasing model. We further demonstrate that these speaker embeddings can capture speaker-related characteristics solely from the phrasing task. Besides, we explore the potential of pre-trained speaker embeddings for unseen speakers through a few-shot adaptation method. Furthermore, we pioneer the application of phoneme-level pre-trained language models to this TTS front-end task, which significantly boosts the accuracy of the phrasing model. Our methods are rigorously assessed through both objective and subjective evaluations, demonstrating their effectiveness. • Speaker-conditioned phrasing model improves accuracy in multi-speaker phrasing tasks. • We explore various speaker embeddings in phrasing models. • We apply phoneme-level pre-trained language models to enhance phrasing accuracy. • We propose a speaker adaptation method for few-shot phrasing tasks. • We verify that speaker embeddings learn human-aligned features via phrasing tasks.