Moral agents as relational systems: The Contract-Based Model of Moral Cognition for AI

As artificial systems are becoming more prevalent in our daily lives, we should ensure that they make decisions that are aligned with human values. Utilitarian algorithms, which aim to maximize benefits and minimize harm fall short when it comes to human autonomy and fairness since it is insensitive to other-centered human preferences or how the burdens and benefits are distributed, as long as the majority benefits. We propose a Contract-Based model of moral cognition that regards artificial systems as relational systems that are subject to a social contract. To articulate this social contract, we draw from contractualism, an impartial ethical framework that evaluates the appropriateness of behaviors based on whether they can be justified to others. In its current form, the Contract-based model characterizes artificial systems as moral agents bound to obligations towards humans. Specifically, this model allows artificial systems to make moral evaluations by estimating the relevance each affected individual assigns to the norms transgressed by an action. It can also learn from human feedback, which is used to generate new norms and update the relevance of different norms in different social groups and types of relationships. The model’s ability to justify their choices to humans, together with the central role of human feedback in moral evaluation and learning, makes this model suitable for supporting human autonomy and fairness in human-to-robot interactions. As human relationships with artificial agents evolve, the Contract-Based model could also incorporate new terms in the social contract between humans and machines, including terms that confer artificial agents a status as moral patients.