Leakage-aware explainable machine learning for highway–rail grade crossing safety screening

Highway–rail grade crossing safety programs should allocate limited inspection and capital improvement resources across more than 200,000 public at-grade crossings in the United States. This paper presents a leakage-aware, explainable machine learning (ML) screening model that identifies candidate high-risk crossings. The model utilizes Grade Crossing Information System (GCIS) inventory records (Form 71) for approximately 20,000 Union Pacific (UP) crossings, merged with accident reports (Form 57) from 2015–2024. A critical methodological challenge is that accident records carry incident timestamps while non-accident crossings do not; naively augmenting both with weather covariates causes the absence of weather data to become predictive of the negative class—a missingness-driven leakage pathway we explicitly address by randomly imputing timestamps for non-accident rows prior to Meteostat weather lookup. An imbalance-aware eXtreme Gradient Boosting (XGBoost) classifier—assessed via the Area Under the Receiver Operating Characteristic curve ( AUROC = 0.887 )—is evaluated at a primary baseline threshold ( t = 0.50 , Recall = 0.860 ). Sensitivity is further analyzed against a validation-optimized threshold ( t = 0.71 , F1 = 0.613 ). Model transparency is provided through SHapley Additive exPlanations (SHAP) attribution and a low-depth surrogate decision tree. Robustness is assessed via a five-variant feature ablation ( AUROC = 0.887 full vs. 0.806 infrastructure-only), a leave-one-cluster-out geographic holdout across four UP network regions (mean AUROC = 0.840 ± 0.023 ), a threshold sensitivity analysis linking operating point to inspection budget, and a cross-railroad generalizability experiment on BNSF Railway ( AUROC = 0.930 , F1 = 0.711 at t = 0.50 ) using the identical pipeline without modification. Results confirm that weather and temporal features contribute meaningfully beyond infrastructure alone, that the model generalizes moderately well across geographically held-out regions, and that the leakage-aware pipeline transfers across Class I freight networks, supporting its use as a data-driven triage tool for crossing safety programs.