Answers are grounded in this paper and cite their sources.
Tool-using LLM agents must decide when to stop gathering information and commit to an answer, but existing benchmarks cannot separate stopping from search skill because the optimal stopping point is unknown. We construct environments in which the sufficiency step k* (the step after which every fact needed to answer has been observed) is known by construction, and in which tool outputs depend only on the step index, so every model sees the same observations and only the stopping decision varies. Across 8 models and 3,526 trajectories we tested five pre-registered predictions that agents are uniformly "sufficiency-blind" over-gatherers. The universal hypothesis is not supported. Reasoning-tuned models (gpt-5-mini, gpt-5-nano, deepseek) over-gather, and gpt-5-mini shows a multi-constraint closure asymmetry (premature-stop ratio 3.43, 95% CI [2.0, 12.5]), the only prediction confirmed, and for one model only. Four smaller instruction-tuned models do the opposite: they stop before sufficiency on 32–64% of solvable tasks, mostly by abstaining, and most often on multi-constraint and open-ended questions; claude-sonnet-4.6 stops near-optimally. No model loops on unsolvable tasks (99% abstain), and budget slack barely changes slack-normalized overshoot (ρ ≈ 0.08), although raw overshoot rises when the enforced budget is loosened. In this setting the costly failure is stopping too early on complex tasks, not the over-gathering and looping that agent evaluation usually targets. The environments, harness and analysis are available from the authors on request. Preprint. Not peer reviewed.
No takes yet. Share an insight, caveat, or question.
Shaheen et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: