Benchmarking study demonstrates low global state fidelity despite high execution success in LLM-agent handoffs, highlighting that task completion poorly certifies underlying transferred state.
Agent benchmarks often treat terminal success as evidence that the state carried across a handoff was adequate. We study which properties of that state a successful execution supports. Our protocol makes the handoff boundary observable: Stage 1 reconstructs a structured state from an event history; an isolated Stage 2 receives only that state and a complete action contract; and a deterministic executor applies the resulting action. Each episode records strict global state fidelity, immediate action support, structured-action correctness, and executed outcome. Across 80 controlled tasks, strict global fidelity is 5.8%, while immediate support is 72.5%. On a matched three-model subset, 164 of 180 executions succeed. Among those successes, strict global fidelity holds in 3.7%, immediate support in 72.0%, and immediate support in 93.8% of state-changing runs. Ignoring unsupported generic links raises success-conditioned global fidelity to 18.3%, still well below local support. Gold-state substitution gives modest downstream gains with task-cluster intervals spanning zero. Deleting one required state bundle from otherwise intact gold state reduces action accuracy from 40/60 to 1/60. A separate 106-pair natural-error repair experiment does not show aggregate improvement; the 43-pair subgroup that restores complete immediate support is also inconclusive, with wide intervals. The results separate three claims that a single success label can blur: global state fidelity, contract-defined support for the current action, and the behavioral effect of changing the supplied state. Successful execution has high local-support coverage on informative transitions, yet remains a weak certificate of the complete carried state.
No takes yet. Share an insight, caveat, or question.
Jiangwei Xue (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: