Improving Pipeline Failure Prediction via Data Sanitization, Hyperparameter Optimization, and Boosting Blending Ensembles

This paper presents a systematic, stage-by-stage methodology for improving AI-based pipeline failure prediction in municipal district heating and water distribution networks. Starting from a baseline Gradient Boosting classifier (ROC-AUC: 0.7243) trained on noisy dispatch logs, we demonstrate the cumulative impact of three improvements: data sanitization via duplicate removal and incident re-linking, single-model hyperparameter optimization via Optuna (35 trials per model), and a weighted blending ensemble combining XGBoost (35%), CatBoost (35%), LightGBM (15%), and Gradient Boosting (15%). The final ensemble achieves a ROC-AUC of 0.8879 on the 2023 independent test set (+16.36% absolute improvement), Precision@20 of 90.0% (18/20 failure sections correctly identified, 5.2× above random baseline), and selective classification accuracy of 96.07% at 78.3% coverage. Bootstrapping analysis (1,000 resamples) confirms statistically significant gains with completely disjoint 95% confidence intervals between baseline [0.692, 0.756] and final ensemble [0.865, 0.908].