Answers are grounded in this paper and cite their sources.
In this editorial, I highlight 3 papers focused on ethics frameworks for artificial intelligence (AI)1–3 which assess bias from the perspective of AI-based applications in health care as well as 2 papers that address the role of AI-based technologies in patient communication4,5 and within the broader context of their studies, consider the potential for AI-based applications to mitigate clinician biases. Chan et al1 conducted a scoping review to examine the real-world impact of AI ethics frameworks across a decade in health care. Eligible studies reported primary research on the qualitative or quantitative impacts of AI ethics frameworks implemented in health care. The search and screening strategies resulted in 16 of 1807 records that met inclusion criteria, comprising 5 preliminary initiatives testing guidelines in practice, 5 case studies, 5 implementation studies, and 1 comparative case study. Narrative synthesis revealed that the frameworks were implemented for 3 key purposes: (1) to develop new AI governance structures and guidelines, (2) as ethical review assessment systems for adopting clinical AI technologies, and (3) as ethical “audit” tools for identifying ethical risks including bias. Real-world impact was predominantly reported through qualitative improvements in process measures (eg, improved trust in AI). Notably, no studies in the review demonstrated a direct link between ethics frameworks and health-related outcome measures such as patient safety. The authors conclude that implementation and impact evaluation of ethics frameworks for AI in health care remain limited. They suggest that stronger evidence is needed to link ethics frameworks with measurable health outcomes. In a Case Study, Koski et al2 report on the processes and recommendations of a multi-stakeholder expert panel (eg, healthcare professionals, AI developers, policymakers) that was tasked with identifying critical issues and formulating consensus recommendations related to the responsible use of real-world data (RWD) in healthcare AI. The panel’s deliberations revealed several critical challenges including: (1) need for data literacy and documentation, (2) identification and mitigation of bias, (3) privacy and ethical considerations, and (4) absence of an accountability structure for stakeholder management. To address these challenges, the panel proposed 15 recommendations in 5 keys areas: (1) adoption of metadata standards for RWD sources; (2) development of transparency frameworks and instructional labels likened to 'nutrition labels’ for AI applications; (3) provision of crossdisciplinary training materials; (4) implementation of bias detection and mitigation strategies; and (5) establishment of ongoing monitoring and update processes. The authors emphasize the importance of guidelines and resources focused on the responsible use of RWD in healthcare AI for developing safe, effective, equitable, and trustworthy applications, as well as the criticality of multi-stakeholder engagement. In a Perspective, Tsoi et al3 share their experiences in developing a responsible AI framework to assess AI solutions at a healthcare organization. The framework includes an intake survey with descriptive information and 21 items aligned with institutional goals to promote fairness (4 items), transparency (7 items), accountability (5 items), and trustworthiness (5 items). At least 5 clinical, analytical, and operational experts rate each of the 21 items on a 5-point Likert scale. The ratings and subsequent consensus recommendation are reviewed by the AI and Automation Advisory group and organizational leadership for final decision making. Of 12 AI solutions evaluated—most in radiology, 10 were conditionally approved. Challenges identified with the process included limited availability of information from vendors to assess fairness and transparency and reliance on volunteer engagement for evaluation of responses. To address scalability and resource constraints, the authors suggest that future iterations of the framework should consider tiered evaluation based on risk likelihood of the AI-based solution. Dellavalle et al4 conducted a mixed-methods study with patient-users of a healthcare system multi-task chatbot integrated with an electronic health record. They surveyed chatbot users, with 20% of 3089 users responding. They also conducted semi-structured interviews with 46 patients who used the chatbot and 2 chatbot developers. The authors analyzed survey data with descriptive statistics and Chi-square tests, interview data with a modified grounded theory approach, and integrated the data with mixed methods approaches. Of relevance to mitigating potential clinician bias, some patients preferred to discuss sensitive topics (such as mental health or gender-affirming care) with chatbots due to less worry about clinician judgment and perceived ability to avoid unpleasant interactions. Patients also preferred chatbots for administrative tasks due to convenience but preferred clinicians for diagnostic tasks. Langevin et al5 conducted remote interviews with 16 primary care practitioners in remote interviews to brainstorm future technologies for improving clinician awareness of implicit bias in patient-provider communication. Participants also completed an online survey in which they rated the priority of educational strategies that could complement the technology. The interview comprised two components. The provocative design component included a short film-based scenario depicting positive and negative provider communication in primary care visits, after which the participants were asked about what they noticed in the scenes and how future technology that monitors clinical communication might help to improve patient-provider interactions. In the second component, participants were asked to reflect on intervention strategies for future communication feedback technology to address implicit bias. The authors performed inductive-deductive thematic analysis of the interview data with implicit bias recognition and management domains as a priori codes. Participants proposed how future technology could improve clinician awareness of implicit bias, but some providers expressed concerns regarding feedback fatigue and the potential impact of technology on reducing time spent with patients. It was also noted that such provider-specific technology-based interventions must occur within an overall organizational context, including education and monitoring. The three papers focused on AI ethics frameworks reflect increased awareness and activity but highlight the need for continued evaluation, and Chan et al. specifically call for action related to linking such frameworks to measurable health outcomes. The latter two studies remind us that humans as well as algorithms exhibit biases. Indeed, several recent papers6,7 in the Journal of the American Medical Informatics Association identified the presence of stigmatizing and privileging language reflective of clinician bias in clinical notes. Consideration of sources of bias and mitigating strategies is critical to advancing the role of AI-based applications in health care. None to declare. None to declare. Not applicable.
No takes yet. Share an insight, caveat, or question.
Suzanne Bakken (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: