Assessing stability and performance of a digitally enabled supply chain: Retrospective of a pilot in Uttar Pradesh, India

BACKGROUND: Immunization supply chains in low resource settings do not always reach children with necessary vaccines. Digital information systems can enable real time visibility of inventory and improve vaccine availability. In 2014, a digital, mobile/web-based information system was implemented in two districts of Uttar Pradesh, India. This retrospective investigates improvements and stabilization of supply chain performance following introduction of the digital information system. METHODS: All data were collected via the digital information system between March 2014 and September 2015. Data included metadata and transaction logs providing information about users, facilities, and vaccines. Metrics evaluated include adoption (system access, timeliness and completeness), data quality (error rates), and performance (stock availability on immunization session days, replenishment response duration, rate of zero stock events). Stability was defined as the phase in which quality and performance metrics achieved equilibrium rates with minimal volatility. The analysis compared performance across different facilities and vaccines. RESULTS: Adoption appeared sufficiently high from the onset to commence stability measures of data quality and supply chain performance. Data quality stabilized from month 3 onwards, and supply chain performance stabilized from month 13 onwards. For data quality, error rates reduced by two thirds post stabilization. Although vaccine availability remained high throughout the pilot, the three lowest-performing facilities improved from 91.05% pre-stability to 98.70% post-stability (p<0.01; t-test). Average replenishment duration (as a corrective response to stock-out events) decreased 52.3% from 4.93days to 2.35days (p<0.01; t-test). Diphtheria-tetanus-pertussis vaccine was significantly less likely to be stocked out than any other material. CONCLUSION: The results suggest that given sufficient adoption, stability is sequentially achieved, beginning with data quality, and then performance. Identifying when a pilot stabilizes can enable more predictable, reliable cost estimates, and outcome forecasts in the scale-up phase.