Machine Learning Techniques for Accountability

Copyright © 2021, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602 SPRING 2021 47 Our goal, in this short overview article, is to begin mapping the landscape of methods for accountability of artificial intelligence (AI) systems. For our purposes, we shall define accountability as being able to ascertain whether an AI system is behaving as promised, which is necessary for determining blame-worthiness. In the context of a self-driving car, AI system accountability could be a question of safety; in the context of credit scoring, AI system accountability could be a question of fairness. In an algorithmic trading system, the AI system accountability could be a question of performance and robustness to certain shocks. In this overview, we will not focus on any particular objective (such as safety, fairness, or robustness); we believe that defining and refining these objectives for each context is a moral decision that must be made by the public and their representatives, not technologists. Rather, our goal is to begin the process of mapping the categories of methods that one could use to assess whether an AI system is meeting its objectives. Artificial intelligence systems have provided us with many everyday conveniences. We can easily search for information across millions of webpages via text and voice. Paperwork processing is increasingly automated. Artificial intelligence systems flag potentially fraudulent credit-card transactions and filter our e-mail. Yet these artificial intelligence systems have also experienced significant failings. Across a range of applications, including loan approvals, disease severity scores, hiring algorithms, and face recognition, artificial-intelligence–based scoring systems have exhibited gender and racial bias. Self-driving cars have had serious accidents. As these systems become more prevalent, it is increasingly important that we identify the best ways to keep them accountable.