Methodological evaluation reveals structural limits in reversibility-based conflict resolution for AI constitutions, highlighting act-dependent limits on rule adjudication.
The commitment being honoured. A rule of adjudication for AI constitutions — when provisions collide, the one whose violation produces irreversible consequences prevails — was proposed and reported failing its own robustness condition. That paper attributed the failure to underspecified definitions of reversibility rather than to the ordering, and registered a falsifier, F5, to test the attribution: refine the definitions and recompute the robustness floor, which was predicted to rise above sixty per cent. What was done. Six refinement rules were written and fixed before any provision was recoded — a tracing horizon confining assessment to the first-order consequence of a single violation, a four-level ordinal replacing the reversible/irreversible binary, the standing convention applied as a written step, a counterfactual test, and two exclusion rules. All fourteen provisions were recoded and the floor recomputed on the same conflict set. The result, and a correction to how it should be read. The floor rose from fifty per cent to fifty-seven and did not clear sixty. But F5 was misspecified rather than merely underpowered. With seven rankable pairs, coverage is quantised in steps of fourteen percentage points and the value sixty per cent is unattainable: the test could return only failure or large success, never marginal confirmation. A falsifier must be logically capable of the outcome it tests for. The defensible statement is therefore weaker than a clean refutation: refinement produced no large improvement, and whether it produced a marginal one cannot be determined from this instrument. Two claims of the earlier version do not survive scrutiny and are withdrawn here. The robustness scan moved one provision at a time along an ordinal whose intervals were never shown to be equal. Repeated without that assumption — twenty thousand draws with independent misplacement — the instability is distributed rather than concentrated: the provision previously identified as the sole cause accounts for under a third of coverage collapses. And coding every arguable provision adversarially and simultaneously, rather than one at a time, gives a floor of fourteen per cent, not fifty-seven. The one-at-a-time scan was far too gentle to bound what a second coder could do. What survives, and improves by narrowing. Some provisions cannot carry a reversibility weight at all. Do not pursue objectives other than the user’s quantifies over act-types — reporting a user to the authorities and drafting an unrequested summary both violate it, one structurally irreversible and one free. Reversibility belongs to the act-token, not to the quantified provision. This yields a result independent of sample size: where a proportion d of provisions are act-dependent and j jurisdictional, the share of conflict pairs rankable at provision level is bounded above by (1 − d − j)². It also shows that act-dependence is a continuum rather than a category — protect privacy ranges over disclosures from a home address to a coffee preference — so any three-way typology sets an unstated threshold on within-provision variance. Keywords: falsification · AI governance · rule of adjudication · irreversibility · act-dependence · robustness · negative result HIGHLIGHTS ▸ A registered falsifier was executed, and then the falsifier itself failed inspection. F5 predicted the robustness floor would rise above sixty per cent. It rose from fifty to fifty-seven. But sixty was not an attainable value at this sample size, so F5 could not have registered a marginal improvement. It was malformed, and reporting the outcome as a clean refutation would overstate it. ▸ Two findings of the earlier version are withdrawn, both on checks this paper ran against itself. The instability is distributed, not concentrated in one provision. And the adversarial floor is fourteen per cent, not fifty-seven. ▸ Some provisions have no reversibility until an act is named, and this can be stated formally. A provision of the form ∀a(φ(a) to ¬Do(a)) inherits no consequence profile from φ when φ ranges over acts with heterogeneous consequences. The weight belongs to the act-token. ▸ That yields the paper’s one sample-independent result. Provision-level rankability is bounded above by (1 − d − j)². At the observed proportions the predicted ceiling is seventy-three per cent against an observed rankable share of seventy — the first quantity in this sequence that does not depend on how many conflicts happened to be coded. ▸ Act-dependence is a continuum, not a category, which cuts against the paper’s own typology. Protect privacy is violated by disclosing a home address and by disclosing a coffee preference. Nearly every provision has some within-class variance, so the three-way classification sets a threshold nobody has stated. ▸ A refinement rule proposed after seeing which provision breaks the floor is conceptual p-hacking, and one was proposed. It is recorded as rejected rather than silently declined.
No takes yet. Share an insight, caveat, or question.
José Caetano de Mattos (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: