Retrospective Validation of a Stage 5 Decision Support Algorithm Using Clinical Trial Data From Stage 4 Smart Insulin Pens

Background: We evaluated whether the frequency of algorithm-generated coaching events, a decision support system (DSS) integrated with a smart insulin pen (SIP), differed according to glycemic outcomes, thereby establishing the algorithmic foundation. Methods: This retrospective analysis evaluated continuous glucose monitoring (CGM) and SIP data from a 12-week randomized controlled trial (NCT06406439) involving individuals with diabetes managed with multiple daily injections (MDI). A total of 338 biweekly CGM profiles from 58 participants were analyzed. For each 14-day period, the coaching event frequency was calculated across 9 predefined coaching categories. Results: Frequencies of hyperglycemia-related coaching (time in range [TIR]: 18.8 vs 34.8; time above range [TAR > 180 mg/dL]: 18.3 vs 33.8) and missed bolus events (TIR: 7.6 vs 11.2; TAR: 7.0 vs 11.1) were significantly higher in profiles not meeting hyperglycemia-related CGM targets (all P < .001). Both coaching categories demonstrated good discrimination for TIR and TAR target achievement (all area under the curve [AUC] > 0.7). Frequency of missed basal detection also showed significant associations with achievement of TIR and TAR targets (all P < .05). The hypoglycemia coaching group effectively discriminated the absence of nocturnal time below range <54 mg/dL (AUC = 0.740). Basal insulin titration coaching frequencies were associated with nocturnal mean glucose changes (all P < .005). Real-time insulin and correction factor reduction recommendations were independently associated with deterioration in TIR and TAR metrics (all P < .05). Conclusions: The frequency of algorithm-based coaching events served as a robust indicator of glycemic control. These findings validate Stage 5 SIP-integrated DSS and suggest its potential utility in identifying unresolved glycemic issues in individuals treated with MDI.