Design-Develop-Decide: A Framework for User and Implementation Evaluation in Digital Health.

Abstract Purpose: This paper introduces the Design-Develop-Decide (3D) framework, addressing the transdisciplinary research gap in the convergence of design think-ing and implementation, particularly in early-stage translation research for health settings. Focused on clinical decision support tools using artificial intel-ligence (AI), the framework emphasizes factors like clinical utility, safety, and human-computer interaction, with a specific focus on in-hospital implementation. Methods: The case study focuses on a clinical AI in-hospital tool for sepsis risk assessment, utilising near real-time data. Employing iterative design evalu-ations and integrating user-design with early implementation science evaluation, the research enhances the usability of digital health tools, specifically in an emer-gency waiting room context. Evaluation methods include in-hospital observations, surveys, Think Aloud usability testing, and various questionnaires. Results: Phase 1 identifies key elements affecting sepsis risk identification, lead-ing to the design goal of supporting clinicians in promptly identifying at-risk patients not assigned to the correct triage category. Phase 2 involves pre-deployment surveys and usability testing, showing improved decision-making in targeted clinical tasks related to sepsis risk assessment. Phase 3 reveals clini-cians’ preferences for integrating the sepsis dashboard into existing workflows, with clinician engagement and understanding through interaction. Conclusion: We advocate for the 3D framework’s integration of formative interaction design methodologies and implementation science, offering practi-cal guidance for early-stage digital health technology adoption. Reflections on strengths and limitations of the approach are provided, highlighting the frame-work’s adaptability beyond clinical decision support tools to other digital systems in the digital health domain.