Retrieval-augmented Generation (RAG): What is There for Data Management Researchers? A discussion on research from a panel at LLM+Vector Data Workshop @ IEEE ICDE 2025

Large language models (LLMs) enable the state-ofthe- art in language processing by framing diverse tasks- from code synthesis and healthcare to finance, digital assistance, and scientific discovery-as next-token prediction problems 38, 53, 60, 65, 72, 20, 68, 76, 32. In addition, LLMs enable automation in data science and engineering, optimizing processes such as data analysis, manipulation, querying, interpretation, research, and education 33, 7, 8, 22, 24, 42, 77, 37, 43, 44, 66, 49. LLMs encode probabilistic token patterns instead of maintaining explicit knowledge structures, which (1) constrains multi-step reasoning under the next-token prediction paradigm; (2) ties outputs to static, pre-cutoff training data-undermining performance on evolving knowledge tasks; and (3) lacks a built-in factual verification mechanism, resulting in hallucinations 25.