Requirements and Attitudes towards Explainable AI in Law Enforcement

Decision-making aided by Artificial Intelligence in high-stakes domains such as law enforcement must be informed and accountable. Thus, designing explainable artificial intelligence (XAI) for such settings is a key social concern. Yet, explanations are often misunderstood by end-users due to being overly technical or abstract. To address this, our study engaged with police employees in the Netherlands, who are users of a text classifier. We found that for them, usability and usefulness are of great importance in explanation design, whereas interpretability and understandability are less valued. Further, our work reports on how design elements included in machine learning model explanations are interpreted. Drawing from these insights, we contribute recommendations that guide XAI system designers to cater to the specific needs of specialized users in high-stakes domains and suggest design considerations for machine learning model explanations aimed at domain experts.