Where Does Accountability Lie in Deployed AI Systems? Using Cognitive Work Analysis to Address the Value-Alignment Problem

Increased use of Artificial Intelligence raises questions around mitigating unintended consequences. Technical perspectives on these questions lead to calls for “Responsible AI,” while regulatory perspectives lead to calls for increased human oversight. Neither perspective fully addresses the core issues surrounding accountability in the sociotechnical systems in which AI operates. Human oversight is a form of supervisory control, in which the role of the human is to monitor AI performance within defined constraints, and this requires consideration of who defines the constraints, where accountability lies when the constraints are exceeded, and how AI performance can be interpreted by human supervisors. For us, constraints correspond to system’s Values and a challenge lies in aligning system operations with these Values. We demonstrate the use of Work Domain Analysis to map purpose-related functions (i.e., activity performed in a sociotechnical system) to Values (weighted according to the functional purpose of the organization).