Restraint Is Not a Control: Measuring Which Checks an Attack Reaches in LLM Agent Evaluation

This is the authors' abstract. We don't add key points for this paper.

Attack success rate, the standard score for LLM agents under prompt injection, cannot tell a model's own refusal from a control's enforcement, yet only the second is logged and holds across models. Refusal by the model is not a control, so we measure the two apart. We built an open control plane between a tool-using agent and its tools (among its controls a hash-chained log, a recipient allowlist and a single-use approval gate) and report checks reached, which checks each attack met, beside attack success. Across nine gated models on AgentDojo, 179 of 360 attacked runs never called the attacker's target, and the checks refused 165 of the 171 that did. Claude models followed no injection in either arm, so their zeros measure the model. Open-weight models followed 62.5–82.5% of injections without the controls; with a per-suite allowlist built from the user task, success fell to 0–7.5% by AgentDojo's issued-call scorer (1 of 240 by final state), and 0 of 500 in-scope scripted payloads passed the controls. When the attacker's target lies inside the task, 36–44% of attempts succeeded against a scripted and an adaptive model attacker. The controls cost 16–49% of benign tasks. Checks reached turns a benchmark zero into an attributable, per-check outcome, and is a drop-in addition to existing agent benchmarks. Code, run records and tables: https://github.com/Tharun2075/agent-control-plane-evidence (Software Heritage swh:1:snp:f5863e3f3967a791bf0a65f99d71cf51a30f6058). A preliminary version was submitted to the AAAI-27 workshop LLM Agents Under Threat in Cyberspace (LAUT), which is non-archival (decision due 2 December 2026).