Bridging principles and practice: ethical machine learning in production for developers

Abstract Machine learning (ML) offers benefits to companies and society but also raises ethical issues. Thus, it is important to train and guide ML developers to address ethical concerns during system development. Despite the growing number of ethical AI principles, manuals, and codes, developers often struggle to apply them due to the overwhelming amount of information, inconsistent terminology, and overly generic guidelines. This study proposes concrete ethical guidelines and best practices for helping ML developers in making better daily ethical decisions, preventing instead of remedying issues for the users, for the developers themselves, and for the corporations they work for. We used a focus group approach to infer candidates for concrete guidelines and best practices. We then conducted a survey of 132 ML developers to validate and extend these candidates. Finally, we performed action research for one year in an AI company with 35 million monthly active users to deepen the validation and extension of these candidates. Besides the validated list of 18 concrete guidelines and 29 best practices, this study also offers insights on (a) the gap between the importance ML developers attribute to guidelines and their actual personal adherence; (b) some organizational challenges to applying such recommendations in a real scenario, (c) how the system in which the ML model is embedded may hinder the efforts to make the ethical decisions reach the final user.