Answers are grounded in this paper and cite their sources.
Machine-learning forecasts of financial returns are usually judged by whether they beat other models, but in low signal-to-noise settings such comparisons can mislead. We propose an evaluation protocol for neural return forecasts that combines four elements: a zero-return benchmark with loss-based block-bootstrap tests and confidence bounds on the panel mean out-of-sample R²; an exact decomposition of R² into squared forecast–return alignment and an amplitude penalty; calibration that uses only past data; and the certainty-equivalent return of a mean–variance investor after transaction costs. We apply it to N-BEATS, N-HiTS and N-BEATSx on daily returns of 19 commodity futures from 2015 to 2026. No neural model improves significantly on zero, and the upper 95% bound on the mean daily R² is 0.16%. Training one network on all commodities raises the mean R² by 3.32 percentage points relative to separate training, yet almost all of this gain comes from smaller forecasts rather than a detectable improvement in alignment, and past-only calibration reduces it to 0.06 points. Sector-based pooling does no better than random grouping. Several forecasts raise the investor’s certainty-equivalent return over a historical-mean benchmark before costs, but none beats equal-weight buy-and-hold, the gains vanish at one-way costs above about 4 basis points, and the training strategy with the lowest loss is not the one with the highest certainty-equivalent return. Lower forecast loss relative to another model should therefore not be read as evidence of predictability or of economic value.
No takes yet. Share an insight, caveat, or question.
Navid Goodarzi (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: