Answers are grounded in this paper and cite their sources.
Large Language Models (LLMs) demonstrate exciting abilities to generate fluent text and perform a variety of Natural Language Processing tasks. However, they remain primarily token-based statistical models, which sometimes struggle to execute higher-level reasoning or to capture abstract ideas within text. This paper surveys state-of-the-art methods that extract the quintessence, or core meaning, from a corpus of text. We consider this first step essential for elevating LLMs from what is defined as Level 0 cognition, purely probabilistic methods of token mapping, to Level 1 cognition, based on thoughts and capable of modelling abstract ideas. We motivate the need for a text-to-thought mechanism, which converts text into conceptual units of abstract thought for downstream statistical modelling, and translates the units back into text. We review foundational techniques such as text summarization, topic modelling, keyphrase extraction, concept extraction, named entity recognition, relation extraction, event extraction, semantic role labelling, semantic parsing, topology learning, text simplification, and content distillation in order to assess their relevance to text-to-thought, identify possible candidates, and discuss challenges in evaluation and implementation. Our findings suggest that the transition to idea-based embeddings has the potential to mitigate common limitations of current LLMs, such as hallucinations and inconsistencies in reasoning.
No takes yet. Share an insight, caveat, or question.
Chihaia et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: