Conditional maximum likelihood estimation for improving annotation performance of n-gram models incorporating stochastic finite state grammars

Language models that combine stochastic grammars and N-grams are often used in speech recognition and language understanding systems. One useful aspect of these models is that they can be used to annotate phrases in the text with their constituent grammars; such annotation often plays an important role in subsequent processing of the text. In this paper we present an estimation procedure, under a conditional maximum likelihood objective, that aims at improving the annotation performance of these models over their maximum likelihood estimate. The estimation is carried out using the extended Baum-Welch procedure of Gopalakrishnan et.al. We find that with conditional maximum likelihood estimation the annotation accuracy of the language models can be improved by over 7% relative to their maximum likelihood estimation.