A structured approach to modifying an implementation package while scaling up a complex evidence‐based practice

Abstract Objective To describe a structured, iterative, data‐driven approach for modifying implementation strategies for a complex evidence‐based practice during a nationwide scale‐up initiative. Data Sources and Study Setting We scaled‐up implementation of Critical Time Intervention (CTI)—an evidence‐based case management model—across 32 diverse community‐based Veterans Affairs (VA) “Grant and Per Diem” case management (GPD‐CM) agencies that serve homeless‐experienced Veterans transitioning to independent living. Primary data were collected using qualitative methods. Study Design We embarked on a scale‐up initiative while conducting a pragmatic randomized evaluation using a roll‐out design, comparing two versions of a CTI implementation package tailored to VA's GPD‐CM program. We iteratively assessed contextual factors and implementation outcomes (e.g., acceptability); findings informed package modifications that were characterized using the Framework for Reporting Adaptations and Modifications to Evidence‐based Implementation Strategies. Data Collection Methods We conducted semi‐structured interviews with Veterans, GPD‐CM staff, and liaising VA clinicians; periodic reflections with liaising VA clinicians and implementation team members; and drew upon detailed meeting notes. We used rapid qualitative methods and content analysis to integrate data and characterize modifications. Principal Findings After each scale‐up wave—in response to variations in agency‐level characteristics— we made iterative modifications to the implementation package to increase CTI adoption and fidelity across the diverse contexts of our scale‐up sites. Modifications included adding, deleting, integrating, and altering the package; core package components were preserved. Conclusions Implementation packages for complex evidence‐based practices undergoing scale‐up in diverse contexts may benefit from iterative modifications to optimize practice adoption with fidelity. We offer a structured, pragmatic approach for iteratively identifying data‐driven, midstream implementation package adjustments, for use in both VA and non‐VA scale‐up initiatives. Our project demonstrates the importance of assessing for and making modifications in a scale‐up initiative, as well as the trade‐offs of projects having simultaneous formative and summative evaluation aims.