Predicting the Future: A Connectionist Approach

We investigate the effectiveness of connectionist architectures for predicting the future behavior of nonlinear dynamical systems. We focus on real-world time series of limited record length. Two examples are analyzed: the benchmark sunspot series and chaotic data from a computational ecosystem. The problem of overfitting, particularly serious for short records of noisy data, is addressed both by using the statistical method of validation and by adding a complexity term to the cost function ("back-propagation with weight-elimination"). The dimension of the dynamics underlying the time series, its Liapunov coefficient, and its nonlinearity can be determined via the network. We also show why sigmoid units are superior in performance to radial basis functions for high-dimensional input spaces. Furthermore, since the ultimate goal is accuracy in the prediction, we find that sigmoid networks trained with the weight-elimination algorithm outperform traditional nonlinear statistical approaches.