Answers are grounded in this paper and cite their sources.
Observational analysis improved machine learning model predictions in NICU settings, suggesting human-centered design methods enhance model development and evaluation.
Human-centered design (HCD) methods in machine learning generally focus on workflow, user interfaces, and data visualizations, but there is the potential to apply these methods to inform the model development and testing process.This study aimed to demonstrate the potential of HCD methods to support the design and testing of machine learning models developed for clinical decision-making.In preparing for formative user testing of clinician facing representations of a machine learning model for detecting sepsis in neonatal intensive care unit (NICU) patients, we discovered that interactive low fidelity mockups using real patient data revealed potential model anomalies. To further investigate these potential anomalies, we utilized the qualitative analysis of interviews with 31 NICU clinicians concerning their experience with neonatal sepsis. The review process was conducted by a multidisciplinary team with members having expertise in neonatology, informatics, data science, and human computer interaction (HCI). Anomalies identified via the mockups and interview analysis were further analyzed by inspections of patient charts and model features and code.The HCD-facilitated review revealed anomalies in three categories: (1) feature inclusion and exclusion, (2) feature importance, and (3) model stability over time. Data entry errors in the electronic health record and their impact on model output were also noted. The review resulted in 41 changes to the model.The discovery of over 41 opportunities to improve our prediction model was a serendipitous by-product of the HCD process. Our results suggest that HCD can be applied not only to model display design and measures of explainability, but to the development and evaluation of the model itself. This case report also demonstrates the need for a multidisciplinary team of clinicians, data scientists, and HCI experts in identifying and addressing issues involving machine learning model performance.
No takes yet. Share an insight, caveat, or question.
Karavite et al. (2025) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: