Exploring performance effects of speaker de-identification for privacy protection on emotional speech using the SPEAKnorm system

In speaker de-identification, much attention has been given to inter-speaker variability. Variability within a single speaker, especially for privacy protection, remains relatively underexplored, with little attention given to the effect of emotion on speaker de-identification. In this study, we present a series of experiments to assess and quantify the effects of de-identification on emotional speech. We employ our previously formulated CRSS-SPEAKnorm system for initial de-identification. De-identification is accomplished by extracting spectral features and F0 from the speech signal and transforming. F0 is altered using Time-Domain Pitch Synchronous Overlap and Add (TD-PSOLA). Spectral feature dimensions are altered using a ML convolutional neural network that maps emotional speech from source speaker to same target speaker in neutral, or source speaker emotional speech towards an average of speakers from an in-set training group. Voice de-identified samples generated were compared to original samples for intelligibility based on WER using an automatic speech recognition system and estimated speech quality using an MOS score estimator. By exploring the effects of these two approaches for quality and intelligibility based on WER and estimated MOS scores, we establish a baseline for emotion suppression in speaker de-identification, as well as identifying areas of improvement in future research directions.