A Comparative Study of Transformer-Based and Classical Models for Financial Time-Series Forecasting

This study compares classical and deep learning models (ARIMA, Random Forest, RNN, LSTM, CNN, and Transformer) for forecasting one-day-ahead log returns rt+1=ln(Pt+1/Pt) using daily data for six U.S.-listed equities (NVDA, TSLA, SMCI, GOOGL, PYPL, SNAP) from 2014 to 2024. Predictors include lagged price/return information, lagged macroeconomic variables (CPI, policy rate, GDP) to reflect information availability, and technical indicators (SMA, RSI, MACD) computed using rolling windows ending at day t to avoid look-ahead bias. Performance is evaluated in a walk-forward out-of-sample design, with hyperparameters selected using time-series validation within each training window. Empirically, results are asset-dependent: ARIMA and Random Forest remain strong baselines; deep learning models show asset-dependent performance, with LSTM occasionally competitive in some settings, and the Transformer competitive but not uniformly dominant. For context, this study also reports a rule-based SMA(10/50) crossover benchmark evaluated net of transaction costs. Overall, the findings suggest that predictive signals in daily equity returns, when present, are modest and must be assessed under strict leakage controls and realistic evaluation protocols.