Auditing information leakage and predictive signal in financial machine learning: a controlled study

This study audits a financial machine learning pipeline designed to detect information leakage, the absence of predictive structure, and the recovery of a known signal. The evaluation uses a controlled financial benchmark comprising 1,304 business-day dates, nine sector-specific asset series, and seven economic indicators generated through a prespecified GARCH-like conditional-variance process with controlled factor exposures. Three base models, a long short-term memory network (LSTM), a simplified attention model, and XGBoost, together with a fixed-weight ensemble are evaluated after correcting an initial configuration in which contemporaneous predictors were mechanically related to the target. All cross-model comparisons use a common sequential asset-date test subset ( N = 1, 476). Under the leakage-free, null-signal condition, the evaluated configurations yield mean R 2 values between −0.0052 and 0.0000 and directional accuracies between 49.53 and 49.95%. Moving-block bootstrap intervals for directional accuracy, based on blocks of consecutive test dates while retaining all assets within each sampled date, include both the balanced 50% reference and the observed majority-class rate for all evaluated models. Explicit sample accounting distinguishes the raw tabular test partition ( N = 1, 755 asset-date rows), the complete-case tabular sample used for XGBoost/SHAP analysis ( N = 1, 746), and the sequential subset used for cross-model comparison. A 10-seed neural-network sensitivity analysis confirms that the LSTM and simplified attention results remain centered near null performance. In a separate controlled validation, a linear dependency is introduced between the standardized first difference of the Chicago Board Options Exchange Volatility Index (VIX) at day t and the target return at day t +1. Across 10 seed-matched repetitions per signal-intensity level, mean R 2 increases from −0.0028 under the negative control to 0.9031 at the highest intensity. The normalized SHAP importance of the injected predictor increases monotonically from 4.08 to 83.04%. The contribution is methodological: the study establishes a traceable procedure for diagnosing leakage, delimiting the scope of inference, and verifying predictive-signal recovery under known data-generating conditions.