Research Progress of RNN Language Model

Language model is the cornerstone of natural language processing (NLP). The language model based on recurrent neural network (RNN) has become the most widely used language modeling technology because of its strong ability of self-learning and analysis of sequence data. In this paper, RNN language model is combed in detail. Firstly, the development history and application of RNN language model are briefly summarized. Then, the general mathematical description form of language model, the basic structure and calculation process of classical RNN language model are introduced. Then, the research progress of RNN language model in computational complexity and model performance improvement is mainly introduced. Finally, the future research of RNN language model is discussed.