A constrained joint optimization method for large margin HMM estimation

In this paper, we propose a new optimization method, i.e., constrained joint optimization method, to solve the minimax optimization problem in large margin estimation (LME) of continuous density hidden Markov model (CDHMM) for speech recognition. First, we mathematically analyze the definition of margin and introduce some theoretically-sound constraints into the minimax optimization to guarantee the boundedness of the margin in LME. Moreover, we propose to solve this constrained minimax optimization problem by using a penalized gradient descent algorithm, where the original objective function, i.e., minimum margin, is approximated by a differentiable function and the new constraints are cast as penalty terms in the objective function. The new method is evaluated in a speaker-independent E-set speech recognition task by using the OGI ISOLET database. Experimental results show that the new constraints are very effective to ensure the convergence of the minimax optimization and the large margin estimation via the resultant optimization method can achieve significant word error rate (WER) reduction over the conventional HMM training methods, such as MLE and MCE.