Evaluating Feature Importance for Speaker Separation Models

Computational speaker separation, or multi-talker separation, attempts to use neural network models to separate a stream containing multiple speakers into an individual stream for each speaker. Such separation has advantages in many situations, for example, an emergency 911 call may contain multiple people talking simultaneously; in a call-center scenario, the agent may be presented with an audio stream that contains a customer speaking while a television is playing in the background; interviews and political debates often consist of speakers talking over each other. In all of these cases, separating the mixed stream into individual streams enables back-end processes to detect which particular stream is of importance, and provides for greater conversational intelligence from the separated streams when compared to the mixed stream. In this paper, we examine which features are important to the speaker separation problem; we comprehensively examine 14 characteristic features of a mixed audio stream to determine the subset of features that lead to a cleaner separation. Our key results are that four features, namely minimum intensity, minimum pitch, difference in intensity, and difference in pitch lead to superior speaker separation. By providing significant evidence into the relationship between input audio features and separation efficacy, this work contributes towards optimizing novel strategies for speaker separation systems.