Retrieval-augmented generation (RAG) leverages LLMs to automatically find relevant documents and to provide users with direct answers to their queries. Supplying retrieved documents deemed to be relevant during answer generation reduces LLM "hallucinations." However, as generative AI proliferates, a RAG system will increasingly find documents that have been generated as well. Although they can be useful, we observe that LLMs favor generated over authentic content during retrieval. In our research, we have shown that detecting LLM-generated content proves to be inherently more difficult than expected. Likewise, citing a source for a generated statement makes it verifiable. But it often remains unclear which sources have been discarded in favor of the cited one. We demonstrate this kind of framing in the context of product search. Direct answers often narrow the user's perspective of the available diversity of choices. The sincerity of a RAG system must therefore be judged by the authenticity of its answers. Date: 18 July 2025.
Martin Potthast (Mon,) studied this question.