Answers are grounded in this paper and cite their sources.
Shopping agents operate in dynamic environments where inventory, prices, promotions, and customer goals can change mid-interaction. Existing evaluations emphasize final outcomes, missing whether agents detect changes, avoid stale actions, and recover. We introduce an environment-grounded verifier that checks agent behavior against evolving store states, customer goals, and tool histories. We benchmark nine open-weight agents in static shopping and three under controlled changes to stock, quantity limits, prices, promotions, and customer goals. The verifier tracks exposure, stale actions, adaptation, and recovery time. Fault injection yields 0.98 F1 with no false alarms, while adaptation verdicts agree with a blind audit in 92 of 100 trials. Under prompted revalidation, agents adapt in 72–84% of trials, but frequently fail to disclose changes: they handle nearly every price increase while rarely mentioning it. Removing the customer’s request to recheck the cart has modest overall effects but can sharply reduce price-change adaptation. For comparable customer-initiated swaps and quantity reductions, every agent adapts in every trial, 16–25 percentage points more often than for matching store changes. Relative to matched unchanged-store controls, store changes lower goal achievement by 16 and 19 percentage points for two agents, with no clear effect for the third. Adaptation does not ensure task completion, and final-cart success can mask stale actions. Thus, evaluating agent reliability requires tracking responses to changes, not just final outcomes.
No takes yet. Share an insight, caveat, or question.
Adiasor et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: