From Attestation to Evidence: Assuring That Human Oversight of AI Works

Regulators, standards bodies, and organizations increasingly require human oversight of high-risk AI systems, but the evidence offered that oversight works is usually a policy, a documented design, or a statement that a human reviews outputs. This paper argues that such attestations show oversight is intended, not that it functions, and that a good decision outcome does not show it functioned either, because an accurate system can produce good outcomes even when reviewers are not catching errors. It proposes a five-level evidence hierarchy for assurance: policy and attestation, documented design, process records, condition indicators, and measured performance. Each level answers a different question, and only the highest level tests whether reviewers detect and correct errors in practice. The paper ties the expected level of evidence to the risk tier of the system, aligns the hierarchy with the EU AI Act (Articles 13, 14, and 26), the NIST AI Risk Management Framework, and ISO/IEC 42001, and shows through a worked example of two insurers with identical attestations but very different measured performance why the upper levels matter. It concludes with guidance for assessors and standards bodies on what to request, what to accept, and where current practice falls short.