Confidence scoring for speech understanding systems

This research investigates the use of utterance-level features for confidence scoring. Confidence scores are used to accept or reject user utterances in our conversational weather information system [10]. We have developed an automatic labeling algorithm based on a semantic frame comparison between recognized and transcribed orthographies. We explore recognition-based features along with semantic, linguistic, and application-specific features for utterance rejection. Discriminant analysis is used in an iterative process to select the best set of classification features for our utterance rejection sub-system. Experiments show that we can correctly reject over 60% of incorrectly understood utterances while accepting 98% of all correctly understood utterances.