Benchmarking study reveals a 75.14% silent state failure rate in stateful LLM agents, indicating that final-answer correctness does not guarantee safe downstream computation.
Stateful LLM agents often expose bounded state for later computation. A correct final answer does not establish that this state is safe, sufficient for continuation, or aligned with the answer. We operationalize this acceptance question with a task-relative, field-sensitive, representation-normalized contract for explicit state carriers. Deterministic adapters map native summary, visible-carry, and typed-state representations into a common claimed-state layer; the contract then evaluates parseability, actionable-state presence, exclusion safety, operational sufficiency, and answer–state consistency. We apply the contract to 7,200 frozen outputs over 40 tasks, five model-provider conditions, six state-transfer protocols, two state budgets, and three runs. Dedicated final-answer success occurs on 696 rows (9.67%); intrinsic state validity on 191 (2.65%); answer-aligned validity on 180 (2.50%); and joint answer–carrier success on 173 (2.40%). Thus, only 24.86% of answer-correct outputs carry answer-aligned valid state, leaving a 75.14% silent-state-failure rate. In the three protocols with estimable valid and invalid strata, valid state is associated with higher next-step success, with task-cluster risk differences of 35.42–72.97 percentage points. Because the separately implemented executor shares task semantics with the validator, these contrasts provide criterion-related operational evidence rather than an unrelated external ground truth. Under this benchmark contract, final-answer accuracy is insufficient when downstream computation consumes the exposed state. All revised scores are replayed offline from saved outputs without model regeneration.
No takes yet. Share an insight, caveat, or question.
Xue et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: