Scope, goals and relationship-building: Tackling the challenges in integrating ethics into AI research

Abstract As artificial intelligence (AI) systems rapidly develop in biomedical research, bioethical frameworks focused on human subjects research are often limited in their capacity to address ethical concerns related to data-driven and black-boxed AI algorithms. Calls to embed Ethical, Legal and Social Implications research into large research consortia focused on AI seek to address the challenges of applying models developed in the context of genomics research to AI projects. Foundational questions related to the conduct of such embedded ethics work include: what are the goals of this work, what counts as “ethics,” who qualifies as an ethics expert and what mechanisms, including organizational structures, can ensure success in achieving ethics goals alongside scientific goals by multidisciplinary research teams? We draw on empirical case studies (including semi-structured interviews and document analysis) of large, federally funded AI research projects in the US that required embedded ethics. Notable findings include the lack of consensus among key players regarding the definition of AI ethics and conflict over the scope of ethics work, which impedes the implementation of ethics-related goals in research practices. These conflicts highlight the differences in disciplinary cultures, contentious boundaries between technical and ethical expertise and the lack of power and impact of the ethics team. Our findings underscore that effective ethics integration depends on trust and respect for ethics scholarship through leadership and relationship-building. We argue that building long-lasting, equitable relationships requires institutional commitment through organizational frameworks and funding mechanisms that prioritize deliberation about ethical concerns, shared values and shared power, as well as co-developing ethics goals alongside science goals. A horizontal approach is needed to nurture joint relationships for fostering ethical and just biomedical AI research and development.