Maximum relative margin estimation of HMMS based on N-best string models for continuous speech recognition

Based on the principle of large margin classifier, recently we proposed two novel training methods, namely large margin estimation (LME) 8 and maximum relative margin estimation (MRME) 9 for speech recognition. In LME or MRME, HMM parameters are estimated to maximize the minimum margin among all training utterances. However their original formulation is limited to isolated-word ASR tasks. In this paper, we propose a new training method based on N-best string models to extend the original MRME framework to continuous speech recognition. We also study a new definition of relative margin which is more theoretically sound than the one used in 9. Experimental results in a connected digit recognition task clearly show that the string-level MRME is very effective in terms of reducing recognition error rates by up to 57% over our best MCE-trained models. A string error rate as low as 0.84% has been achieved on the standard TIDIGITS test set, which is the best result that has ever been reported in this task.