Dynamic, hierarchical, and global representations of semantics based on large language models in real-time Chinese natural language comprehension and their neural signatures

Language is a fundamental tool that humans use to generate, convey, and interpret meaning, with semantics at the core of this process. Understanding how semantic information drives language comprehension in the human brain is a central question in cognitive science and has profound implications for artificial intelligence and clinical applications. However, previous research has often overlooked the hierarchical and continuous nature of semantic processing in natural language comprehension—particularly in Chinese, a language that relies heavily on contextual semantic cues. This project aims to develop an integrative framework bridging computational linguistics, cognitive neuroscience, and clinical neurology to investigate how semantics underpins the comprehension of Chinese natural language. Recent advances in large language models (LLMs) provide new opportunities to quantify semantic factors and construct multi-level, dynamic semantic network models. Specifically, we propose to computationally model semantic relationships along three key dimensions: 1) Serial representation, capturing the real-time dynamics of semantic features in linear input; 2) Hierarchical representation, measuring semantic distance across linguistic constituents at multiple levels; 3) Global representation, indicating information density across an entire discourse. To support this, we will develop a large-scale, text-level model to enhance LLMs’ capacity to extract and compute semantic metrics from naturalistic language input. This fine-tuned model will enable real-time semantic representation analysis across texts, speech, and dialogue, offering immediate feedback for language decoding studies and applications in language evaluation and preservation in patients with brain diseases. This project will combine eye-tracking and high-density EEG to empirically investigate the neurocognitive correlates of semantic processing during real-time language comprehension. A self-paced visual reading experiment and an auditory perception experiment will be conducted. Our preliminary study based on an eye-tracking reading task has demonstrated that LLM-based semantic representations strongly correlate with human eye-movement patterns during natural reading, suggesting a parallel between AI-driven language mechanisms and human comprehension.