Entropy Is Not Enough: Output-Distribution Signals for Detecting Context–Memory Conflicts in Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) systems can be misled when retrieved context contradicts what the model knows, and a cheap inference-time conflict detector would let costly arbitration be applied selectively. We ask whether the output distribution of a language model carries a usable conflict signal. We compare entropy-based signals with identity-aware signals (Jensen–Shannon divergence, argmax disagreement, probability mass moved away from the closed-book answer, likelihood of the closed-book answer under the context). In experiments with Qwen2.5-3B/7B-Instruct on TriviaQA (type-matched wrong-answer contexts) and CounterFact, with a neutral-context control, entropy shift is near chance in three of four settings (AUC 0.46–0.54) and at most 0.65 in the fourth, whereas identity-aware signals reach AUC 0.84–0.95 on CounterFact and 0.51–0.76 on TriviaQA. A conditional analysis shows these signals largely detect that the model was swayed by the context. Downstream, gating on identity-aware signals matches or exceeds always-arbitrating within sampling error while arbitrating only 32–57% of instances, whereas entropy-based gating loses poison resistance; arbitration prompt wording matters more than the gate. Results are limited to two small models, 83–100 test questions per setting, and synthetic conflicts. Code and results are included in the archive. This is a preprint and has not been peer reviewed.