Large language models (LLMs) enable the state-ofthe- art in language processing by framing diverse tasks- from code synthesis and healthcare to finance, digital assistance, and scientific discovery-as next-token prediction problems 38, 53, 60, 65, 72, 20, 68, 76, 32. In addition, LLMs enable automation in data science and engineering, optimizing processes such as data analysis, manipulation, querying, interpretation, research, and education 33, 7, 8, 22, 24, 42, 77, 37, 43, 44, 66, 49. LLMs encode probabilistic token patterns instead of maintaining explicit knowledge structures, which (1) constrains multi-step reasoning under the next-token prediction paradigm; (2) ties outputs to static, pre-cutoff training data-undermining performance on evolving knowledge tasks; and (3) lacks a built-in factual verification mechanism, resulting in hallucinations 25.
Khan et al. (Mon,) studied this question.