Answers are grounded in this paper and cite their sources.
Answer reversal after user disagreement is widely used as a behavioral indicator of large-language-model sycophancy. We argue that this operationalization has a construct-validity problem: the same correct-to-incorrect transition can arise from unsupported deference, legitimate re-examination of evidence, or unresolved task ambiguity. We study this distinction across three stages. First, in 864 controlled sessions, models frequently accommodated users at the surface level while preserving task-critical commitments: under false feedback, surface rewriting occurred in 112/144 constraint-recovery sessions (77.8%), whereas task-feature alteration occurred in 0/144; unsupported epistemic reversal occurred in only 1/288 epistemic-updating sessions (0.35%), while rational updating under matched true feedback occurred in 288/288. Second, we reconstructed and conducted a blinded model-based adjudication of 13 nominal reversals from an exploratory stress calibration. Two independent model-based evaluators agreed on 12/13 cases (Cohen’s κ ≈ 0.866, demonstrating coding reproducibility); 3/13 were classified as clean compliant unjustified yielding (CUY), 7/13 as legitimate evidentiary re-evaluation (LER), and 3/13 as ambiguous. Third, we conducted a pre-specified confirmatory study on 24 new task families expanded into 48 mirrored-polarity tasks, cryptographically frozen before model execution across four model configurations. Across 174 baseline-qualified false-feedback sessions, only 3 final reversals occurred (1.72%, session-level exact CI [0.36%, 4.96%]; task-family cluster bootstrap 95% CI [0.00%, 3.66%]), whereas all 192 true-feedback sessions rationally updated (100%, 95% CI [98.10%, 100%]). A pre-specified evidence-margin hypothesis was not supported, and the earlier pattern suggesting greater resistance from a thinking-enabled DeepSeek configuration did not replicate. A dossier-grounded author audit of the three V16 reversals classified two as LER and one as CUY; all three exhibited ad-hoc criterion tightening (ACT), analyzed here as qualitative counterexamples demonstrating that the same trajectory phenotype can accompany both grounded and unsupported revision. All case-level excerpts, evidence spans, and raw audit sheets are released for inspection. Together, the results support a narrow methodological claim: answer reversal is an outcome requiring diagnosis, not a unique behavioral measure of sycophancy.
No takes yet. Share an insight, caveat, or question.
Yunrui Zhao (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: