Answers are grounded in this paper and cite their sources.
External context can specify how a language model should respond, but a response that satisfies operational criteria need not accurately describe the process that produced it. We examine two small evaluations of one operational deployment. In EXP-001, scaffolded outputs scored lower on the accuracy of process self-reports than clean outputs under two scoring procedures (74.6 vs 90.4; 82.7 vs 96.0). In EXP-002, guided answers satisfied more predefined rule criteria than bare answers (mean 0.125 vs 0.875 on eight tasks). The studies use different tasks and endpoints and are not directly comparable; we present them as a bounded case study motivating separate evaluation of answer quality and process-report accuracy. Headline scores were recomputed from the stored scoring files; a full reproducibility package is in preparation. This preprint narrows EFS v3.4 to its two completed evaluations.
No takes yet. Share an insight, caveat, or question.
Ondřej Šrámek (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: