Heckman-Corrected Epistemic Uncertainty: Selection on Unobservables Defeats Importance Weighting

Training data for machine-learning models is routinely collected by a selection process the modelnever sees: loans are observed only when granted, outcomes only when a test was ordered, regions ofinput space sampled only when someone chose to sample them. The field’s default fixes — importanceweighting, covariate-shift correction, MAR imputation — assume selection is ignorable given observables.Econometrics solved the harder problem in 1979: Heckman’s two-equation model jointly fits a probitselection equation and an outcome equation linked through correlated errors, and the inverse-Millsratio term corrects the outcome model for selection on unobservables — the case where importanceweighting is structurally helpless. We instantiate that machinery for deep epistemic uncertainty: a deepoutcome network, a linear selection head, and a joint bivariate-normal likelihood over all units (observedand not), ensembled for epistemic variance. Our implementations first reproduce the seven-digit Statareference output for both the classic two-step and the maximum-likelihood estimator on the RANDHealth Insurance data (max coefficient deviation 2.8 × 10−6 two-step, 5.0 × 10−7 MLE). In a controlledgenerator where sampling probability depends on an unobservable correlated (ρ swept 0 → 0.9) withthe outcome noise, deep ensembles, MC dropout, and GP baselines are overconfident exactly where datawas avoided for reasons correlated with the outcome (coverage of nominal-90% intervals falls to 64.4% atρ =0.9), and importance weighting with oracle propensities does not fix it (43.1%) — reweighting correctsthe covariate distribution, not the conditional bias E[y | x,s=1] ̸ = E[y | x]. The Heckman-correctedpredictive distribution restores coverage (88.9%) when the selection equation contains an instrument —a variable affecting selection but not the outcome — and degrades gracefully but measurably withoutone (40.3%); we sweep this honesty curve rather than hide it. The joint MLE needs a warm-up scheduleto match the two-step’s stability with deep feature maps (otherwise the flexible outcome net absorbs thecorrection), a methods finding we characterize. On real tabular data with induced, documented MNARselection, the corrected intervals are the best-calibrated (lowest region-ECE) of every non-oracle methodin selected-against regions on both datasets; baselines that match its raw coverage do so only by overwidening everywhere, which region-ECE exposes. A discussion vignette fits the same two-equation modelto public benchmark reporting panels (Papers-with-Code tables), where the no-instrument pathology ourcontrolled experiments quantify appears in the wild: most fits hit the |ˆρ| = 1 boundary. We state plainlywhich identification regime a practitioner is in, and release the faithfulness-gated implementation.