The debate over catastrophic AI risk is looking in the wrong place — notbecause model capability is unimportant, but because capability is only onepart of the path to catastrophe. Pacing can buy time. Evaluations can revealdangerous capability. Regulation and agreements can impose common obligations.But none of those, by themselves, governs what happens when a model becomes anoperating agent. How capable an AI model is determines what it can think up. What it can touchdetermines what it can make happen. This paper separates three risk drivers that require different remedies, andsets out four governable surfaces where capability meets the world: Knowledge,Action, Execution, and Accumulation. Two mechanisms do most of the work —gates that refuse and signals that record. Many of the technical piecesalready exist, built separately for other purposes, and do not yet form acoherent safety system for catastrophic risk. The paper then turns to whatmakes those controls binding: national law, international agreements, and thehuman answerability every layer ultimately depends on. An opinion white paper by Marcos M. Campos, Principal AI Architect atMicrosoft and a founding board member of the US AI Council. Written in apersonal capacity; the views are the author's own.
No takes yet. Share an insight, caveat, or question.
Marcos M. Campos (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: