Natural Language Understanding (NLU) has long been constrained by semantic ambiguity and context dependence, whereas traditional approaches struggle to overcome the limitations of unidirectional encoding and static semantic representation. Through a literature review method, this paper systematically examines the theoretical foundations of BERT (Bidirectional Encoder Representations from Transformers) and its empirical applications in NLU. Grounded in the Transformer encoder and the Masked Language Modeling objective, BERT addresses the limitations of traditional models through bidirectional context modeling, and its fine-tuning mechanism supports efficient downstream task transfer. In the context of practical applications, BERT has improved semantic-matching accuracy in information retrieval and achieved breakthroughs in question-answering systems and the medical domain. In addition, research on the improvement of BERT is continuously advancing, including model compression, multimodal extensions, and innovative pre-training objectives, which continue to advance the models evolution. The article concludes that BERT fills a critical gap in systematic pre-trained-model research and supplies an effective blueprint for industrial-grade NLU systems, but still has limitations in computational cost and long-document processing. Future work should therefore prioritize dynamic sparse training and unified cross-modal architectures.
Zining Zhu (2025) studied this question.