A neural network-hidden Markov model hybrid for cursive word recognition

We present a neural network-hidden Markov model hybrid for the recognition of cursive words which are represented as left-right sequences of graphemes. The proposed approach models words with ergodic HMMs and is designed for small vocabularies. A single neural network provides grapheme observation probabilities for all HMMs in order to compute the most likely word model. During the iterative EM like training of the hybrid, the HMMs provide the targets for the discriminant training of the neural network. An extension of the approach to letter models which can be concatenated in order to form word models and which allow for large vocabularies is also briefly discussed. We report results obtained on a large data base of words from French cheques, showing recognition rates close to 93% for the 30 word vocabulary relevant for French legal amounts.