Blood Glucose Forecasting on OhioT1DM: Evaluation-Protocol Effects, Tree Baselines, and Limited Incremental Predictive Value of Heart Rate

The forecasting accuracy on the OhioT1DM benchmark for blood glucose varies widely between reports, and differences in the evaluation protocols employed limit comparability across studies. Using the six participants of the 2018 cohort, this study held one gradient-boosted tree configuration fixed. The official temporal split served as the reference condition, and selected evaluation choices were compared with matched reference conditions. Replacing that split with a within-patient row-shuffled split lowered the apparent RMSE from 19.4 to 17.0 mg/dL at 30 min, and from 44.8 to 34.0 mg/dL at 120 min. At the 30 minute horizon, 96.9% of shuffled test origins had a same-participant training origin within five minutes. Such adjacent origins shared 11 of the 12 observations in the maximum look-back window. Centered target smoothing lowered scores further against the smoothed target; however, a model trained on smoothed targets predicted raw glucose no better than the raw-target model, so that apparent gain reflected an easier evaluation target rather than better prediction. Under chronological validation, the mean paired RMSE difference between LightGBM and XGBoost was no more than 0.5 mg/dL at any horizon, and all patient-level 95% t-intervals spanned zero. Adding current heart rate, or 24 engineered heart-rate summaries, gave no consistent improvement, including in elevated-heart-rate windows. Selected evaluation choices thus produced larger differences than those between the two tree models. Transparent temporal evaluation, matched preprocessing, defined evaluation masks, and strong fixed baselines are needed to interpret the reported progress on this benchmark.