Substantialism or agnosticism in Moral AI: A reply to Sinnott-Armstrong, Schaich Borg and Conitzer

Sinnott-Armstrong, Schaich Borg and Conitzer (SSC) have previously suggested that artificial intelligence (AI) could help us with moral decision-making in a medical context. Their reply to our paper engaging with their account of Moral AI clarifies several aspects of their view and indicates important overlaps between their position and the one we argue for. In this response, however, we elaborate briefly on three remaining concerns. First, we have concerns about their focus on prediction rather than moral judgement; the help we would like to have from Moral AI may call for a thicker kind of advice. Second, we challenge the plausibility of SSC’s metaethical agnosticism; maintaining neutrality on metaethics while using the metaethical concept of reason seems to us a very delicate balancing act. Third, we wonder about the way in which their approach really is akin to narrow reflective equilibrium, and more specifically, about the role of their approach when it comes to wide reflective equilibrium. The conclusion is that a substantialist approach to Moral AI may be both more theoretically plausible and useful from a practical point of view than a purely predictive model.