The Evolution of Financial Time Series Forecasting: From Classical Econometrics to Temporal Foundation Models and Multimodal LLMs

Financial time series forecasting sits at the intersection of applied mathematics, econometrics, and machine learning. The data generating processes involved are characterized by non-stationarity, regime-dependent volatility clustering, very low signal-to-noise ratios, and reflexive feedback between market participants and asset prices. Over roughly six decades, the dominant methodology has moved through five recognizable phases: classical linear econometrics (ARMA, ARIMA, regularized VAR, VECM), conditional volatility modeling (ARCH/GARCH families, Stochastic Volatility), recursive online signal processing (LMS, RLS, Kalman filters), deep sequence learning (LSTMs, TCNs, patch-based Transformers, State Space Models), and the contemporary period (2024–2026) of Time Series Foundation Models (TSFMs), Temporal Language Models (TLMs), and LLMs adapted for financial reasoning. For the comparative analysis in Section 10, the deep learning phase is split into two paradigms (recurrent/convolutional networks and Transformer-based architectures), and the foundation model phase into numerical TSFMs and multimodal LLM adapters, yielding seven paradigm entries in total. This paper traces that trajectory through the lens of what we call the Capacity-Adaptability Trilemma, a three-way tension between representational capacity, online adaptability, and structural interpretability. We also review evidence on where foundation models fall short in financial contexts, including benchmark contamination and architectural mismatches that can cause large zero-shot models to underperform simple statistical baselines. To address this, we propose Econometrically-Informed Neural Networks (EINNs) as a way to embed econometric constraints into modern deep architectures. We present four self-contained synthetic experiments run on a MacBook Air M1, which offer only limited, illustrative support for these claims. First, the best model class depends on the data regime: a linear AR/ARMA baseline was best on stationary low-SNR data and across all tested SNR levels in our setup, RLS adapted best under parameter drift, and an LSTM was best only on a high-SNR nonlinear process. Second, a noise-dependent crossover exists between DLinear and PatchTST. Third, trained autoregressors and MLPs do not beat naive persistence on synthetic random walks. Fourth, a proposed VECM regularizer keeps forecast spreads stationary in a bivariate cointegrated setup, though it does so by producing spread forecasts with much lower variance than the true spread. All experiments use synthetic data, and validation on real financial series remains future work.