We propose a selective prediction framework for clinical triage that combines calibrated probabilistic modeling, conformal prediction, and cost-aware deferral to prioritize patient safety. The system produces set-valued predictions with finite-sample coverage control and defers low-confidence cases to clinicians when doing so reduces an explicit expected clinical cost. We construct prediction sets using split conformal prediction, a group-conditional Mondrian variant for gender-stratified coverage, and an importance-weighted variant to improve robustness under distribution shift, and we select a single deferral threshold by minimizing expected cost on a held-out calibration set. On temporally separated in-distribution and out-of-distribution test splits for early sepsis prediction, selective deferral yields a favorable risk–coverage trade-off, reducing error on retained cases by 49.6% on the in-distribution (ID) test split and 46.7% on the out-of-distribution (OOD) test split, both at 80% coverage, while achieving low expected cost on both splits with only a moderate increase under shift. Calibration remains strong with low expected calibration error on both test sets, and rule-out performance is conservative, attaining near-perfect negative predictive value at a 95% sensitivity target. Coverage stays close to the nominal 90% target in-distribution and degrades only slightly out-of-distribution, with the weighted method most robust, and the Mondrian method reduces the gender coverage gap to 1.4 percentage points. These results indicate that conformal uncertainty quantification combined with cost-aware deferral can provide transparent and safer clinical decision support that degrades gracefully under temporal distribution shift.
Kwon et al. (Fri,) studied this question.