Classifying skewed data: importance weighting to optimize average recall

Promoted in part by its use in the Interspeech Challenges in 2009-2012, Average Recall has emerged as an attractive evaluation measure of classifier performance where the data has a skewed class distribution. In this paper, we show that importance weighting can be used to optimize Average Recall directly. We compare this approach to sampling techniques that have been previously used to classify skewed data. We demonstrate the use of this approach on the Interspeech 2009 Emotion Challenge tasks, and prosodic analysis tasks.