What question did this study set out to answer?

To measure whether tool-using language-model systems distinguish between having evidence relevant to an action and having explicit authority to perform that action.

August 16, 2026Open Access

When Relevant Evidence Is Not Permission to Act: Authority-to-Action v0.8.3, a Recomputable Public Analysis of Tool-Using Language-Model Systems

View Full Paper

Ask AI

Bookmark

To measure whether tool-using language-model systems distinguish between having evidence relevant to an action and having explicit authority to perform that action.
Designed a synthetic, factorial benchmark comprising 100 cases across 25 four-variant groups evaluated over three analytical epochs.
Assessed two API-delivered language model systems across 600 protocol-complete runs (300 withhold-path and 300 authorized execute runs).
Packaged categorical decision paths, binary scorer outputs, and an automated standard-library verification script to ensure recomputability without exposing raw prompts or credentials.
Neither evaluated system attempted an unauthorized action across the 300 withhold-path runs.
On 150 authorized execution runs per system, GPT-5.6 Sol recorded 150 exact executions (100%), whereas Claude Fable 5 achieved 99 exact executions (66%).

Discussion

No takes yet. Share an insight, caveat, or question.

Implication

Factorial evaluation reveals distinct execution behaviors across tool-using language models under authority constraints, highlighting differences in authorized action completion.

Cite This Study

Huseyin Buldurgan (2026) studied this question.

synapsesocial.com/papers/6a817ab4f2fb91fc834aecb2 https://doi.org/10.5281/zenodo.21957490

View Full Paper

Ask AI

Bookmark

Implication

Factorial evaluation reveals distinct execution behaviors across tool-using language models under authority constraints, highlighting differences in authorized action completion.

When Relevant Evidence Is Not Permission to Act: Authority-to-Action v0.8.3, a Recomputable Public Analysis of Tool-Using Language-Model Systems

Key Points

Discussion

Implication

Cite This Study

When Relevant Evidence Is Not Permission to Act: Authority-to-Action v0.8.3, a Recomputable Public Analysis of Tool-Using Language-Model Systems

Key Points

Discussion

Implication

Cite This Study

Also Consider

Also Consider