Answers are grounded in this paper and cite their sources.
Across 60 paired ambiguous/clear tasks spanning coding, everyday planning, and customer support, a lightweight, budget-tier agentic model (Gemini Flash Lite)'s self-reported confidence is not merely uncalibrated but inversely related to correctness in the region that matters most: the model was right 100% of the time when it self-reported confidence below 40, and only 60–71% of the time when it self-reported confidence above 80 (overall Expected Calibration Error = 0.633; AUROC for confidence predicting "should have asked" = 0.622, rising to 0.845 in the coding domain and falling to 0.353 in customer support — a figure based on a thin n=3 comparison in that domain; see Discussion). This work introduces a small, reusable benchmark — 60 paired tasks (ambiguous / fully-specified) across three domains — each with a gold clarifying question and a defined "reasonable assumption," and releases the dataset, elicitation harness, and grading scripts so additional models can be evaluated as a follow-up.
No takes yet. Share an insight, caveat, or question.
Niveditha S. Nair (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: