Experimental findings show structured summaries improve reasoning in less capable LLMs, indicating a need for adaptive methods.
LLM-based optimization agents increasingly produce structured reasoning artifacts — hypothesis summaries, causal models, prediction logs — that persist across iterations. The assumption is that forcing articulation regularizes reasoning, as the self-explanation effect suggests it does for human learners. We test this assumption using SynthOracle, a family of synthetic multi-objective optimization oracles with known causal structure that enables separate measurement of optimization quality and reasoning quality. We pair the benchmark with a verbal regularization (VR) protocol that requires the agent to hypothesize, predict, and reconcile at each iteration via a structured summary embedded in conversation context. Through controlled ablations on two oracles and two models (Claude Opus 4.6 and Sonnet 4.6; n = 5 paired seeds per condition), we find that the forced summary is scaffolding, not regularization: it improves optimization for a less capable model (Sonnet, +26%) and a harder task (12-dimensional with noise, +14% to +35%), but hurts the most capable model on the simplest oracle (Opus, -35%, p = 0.0006). The pattern is consistent across 18 of 20 paired seeds (binomial p = 0.0002). A mechanism-isolation experiment shows that the performance penalty comes from the summary's persistence in conversation context, not from the cognitive cost of producing it: an agent that produces the summary but does not receive it back performs identically to one that never produces it (p = 0.35). We connect this finding to the architectural trend in frontier reasoning models toward ephemeral chain-of-thought traces and derive a design principle: match the rigidity of your reasoning protocol to the gap between model capability and task difficulty.
No takes yet. Share an insight, caveat, or question.
Kartik Ganapati Bhat (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: