Dynamic continuous progressive neural networks for evolving streaming time series

Abstract This work addresses the challenges of learning from Evolving Streaming Time Series, a scenario defined in streaming contexts by the need for continuous learning, managing temporal dependence, reacting to concept drifts, and avoiding catastrophic forgetting. Streaming Continual Learning (SCL) is an emerging paradigm specifically aimed at tackling these issues. To this end, SCL combines elements of Streaming Machine Learning, Continual Learning, and Time Series Analysis. The core idea of SCL is to maintain a progressively enriched representation that reflects everything learned so far. When a drift occurs, the model integrates newly emerging information and also selectively draws on earlier knowledge. In this context, the present work proposes Dynamic Continuous Progressive Neural Networks (DYNcPNN), a comprehensive and pioneering SCL embodiment built on our previously introduced cPNN architecture. DYNcPNN is a dynamic variant designed to adapt more effectively to concept drifts. Its key contribution is a mechanism that dynamically determines when to expand the architecture to incorporate new knowledge, reducing unnecessary complexity while preserving high performance. DYNcPNN also introduces a strategy to prevent forgetting when, after a concept drift, the model continues learning without expanding, thereby risking overwriting the existing knowledge. The model integrates an automatic concept drift detection system that enables adaptation without manual intervention. Experimental results show that DYNcPNN consistently outperforms traditional SML models, cPNN, and a continuously trained LSTM. It adapts more quickly to concept drifts, effectively mitigates catastrophic forgetting, and optimizes memory usage. Moreover, the results highlight the limitations of SML models, which are unable to account for temporal dependence.