Artificial power, domination and control of humanity’s future

In the face of rapidly improving AI capabilities, AI theorists have warned about the risks of ceding power to autonomous artificial agents. In particular, going sufficiently far down this path might mean ‘losing control of our future’: an outcome some have ranked as a worst-case catastrophe nearly on par with human extinction. A natural way to motivate this judgment is via the thought that losing human control might lead to our domination by future AI systems. Republican political philosophers such as Pettit have held that domination characteristically involves important kinds of harms, even when the dominating party is benevolent and non-interfering. Is this thought correct? After developing an account of what it means to control humanity’s future, I show that AI systems possessing such control would indeed be likely to dominate humans on our best theories of arbitrary power. However, contrary to republican orthodoxy, I argue that this need not lead to two of the three primary harms traditionally associated with domination: the psychological and social costs of subordination, and the burden of obligatory strategic behavior. Meanwhile, the third harm—pervasive uncertainty about interference—remains a serious concern even in otherwise human-friendly AI power scenarios. These results should be of interest to theoretical political philosophy, as they suggest that artificial power has unique features which are as yet underrepresented in discussions of domination.