Freedman’s Paradox at Forty: A Practitioner’s Guide to Variable Selection Failures and Safeguards

In 1983, Freedman showed that variable selection on pure noise can produce spurious regressions. We benchmark 17 select-and-report workflows from five families across six Gaussian linear scenarios and add two reference-only procedures; four of the 19 provide valid-inference references. On common support, AIC stepwise falsely selects at least one variable in 89.9% of null samples. This risk does not fall as n grows to 5,000; larger samples improve signal recovery. No method family dominates the error–recovery curves across the evaluated cells and prespecified tuning paths. Unlike the search and screening defaults, the safeguards set an error target in advance. Apparent in-sample fit after selection often corresponds to negative out-of-sample R2. Among finite intervals, the valid-reference procedures’ native intervals achieve 94.8% to 95.1% marginal coverage. Under the complete null, conditional-on-selection coverage of naive intervals from the compared selection workflows is at most 7.8%. Workflow choice should therefore follow the goal: inference, prediction, or exploration.