VOCALIS: A Lightweight Neural Adapter for Synthetic-to-Natural Speaker Embedding Adaptation

VOCALIS is a lightweight neural adapter designed to reduce the representation gap between synthetic and natural speaker embeddings in zero-shot speech synthesis pipelines. Modern text-to-speech and voice cloning systems frequently generate synthetic speech that is subsequently processed by speaker encoders trained exclusively on natural speech. This mismatch results in a domain gap that can negatively affect downstream speaker representation quality. Rather than modifying pretrained speaker encoders or speech synthesis models, VOCALIS introduces a small residual adapter that operates entirely in the embedding space, learning to map synthetic speaker embeddings closer to their natural counterparts. The adapter is trained on paired natural and synthetic speech generated from the VCTK corpus using a speaker-disjoint protocol. Experimental evaluation demonstrates consistent improvements over both an unadapted baseline and representative domain adaptation baselines, including Mean Shift and Deep CORAL. The learned adapter also generalizes to synthetic speech produced by an unseen text-to-speech system (gTTS), despite never being trained on that synthesis method. The proposed model contains approximately 462,000 trainable parameters, trains in only a few minutes on standard CPU hardware, and introduces negligible computational overhead during inference. This Zenodo record contains:• The complete research paper (PDF)• The complete implementation of VOCALIS Authors:Balaji Ganesh TammireddyPuruhutika Rao Tata Keywords:Speaker Embeddings, Domain Adaptation, Zero-Shot Speech Synthesis, Voice Cloning, Artificial Intelligence, Machine Learning, Speech Processing, ECAPA-TDNN, Deep Learning.