Invisible Demand Theory: How AI-enabled work erodes the conditions for human oversight

Organisations deploy artificial intelligence (AI) on the assumption that human workers will detect and correct its errors. This article proposes Invisible Demand Theory (IDT) to explain why the capacity for that oversight is withdrawn while it is working, and why the withdrawal is not corrected. IDT builds on established accounts of invisible prevention work, drift, ritual verification and attribution after failure, and specifies what is distinctive when the object of oversight is an AI system. Its foundation is asymmetric visibility: when oversight succeeds, its product is a non-event that organisations do not count; when it fails, its product is a visible error attributed to the person nearest the decision. IDT’s core mechanism, the self-concealing loop, specifies what this asymmetry does when the object of oversight is AI. Because errors caught before use leave no record, an organisation’s measure of AI reliability is partly produced by the oversight it is used to evaluate. Rising measured reliability justifies withdrawing oversight, and withdrawing oversight censors the evidence that it was needed. Two properties of generative AI deepen the loop. Fluent output hides the demand from workers as well as organisations (double invisibility), and material model changes can make historical failures an unreliable guide to current oversight requirements (unlearnable demand). IDT separates the visibility of a demand from its resourcing and specifies two demands (verification labour and calibration labour), a state (vigilance debt) and a process (accountability absorption) through which visible failures are converted into individual blame rather than renewed resourcing. It derives nine propositions, each stated so that it can distinguish IDT from the theories it inherits from. In Aotearoa New Zealand, cultural verification labour adds to the cultural double-shift carried by Māori staff, and measured reliability is most inflated for content involving te reo Māori and mātauranga Māori. The presence of a human approver is not evidence of human oversight.