Abstract Complex data streams are highly dynamic, large-scale, and prone to continuous distributional shifts, posing significant challenges for neural network-based classification systems—particularly in maintaining accuracy and efficiency without frequent retraining. To address these issues, this study proposes an intelligent incremental learning framework that integrates a Dynamic Grasshopper Optimization Algorithm (DGOA) with a Multilayer Perceptron (MLP) neural network for real-time hyperparameter optimization. The proposed system functions as an adaptive intelligent system, utilizing DGOA—an enhanced form of the traditional Grasshopper Optimization Algorithm equipped with dynamic parameter control and online swarm reconfiguration—to autonomously adjust to evolving data characteristics. Unlike conventional GOA, the dynamic variant modifies its search behavior and population dynamics in real time, enabling continuous learning without restarting the optimization process. Through this mechanism, the model incrementally tunes critical hyperparameters such as the learning rate and momentum, resulting in improved accuracy and generalization on unseen data. The main contribution of this research lies in developing a fully online, swarm-intelligence-driven hyperparameter optimization strategy tailored for big data streams. Experimental evaluations on the Australian electricity market dataset demonstrate that the DGOA-based MLP achieves a classification accuracy of 89.5%, outperforming Grid Search (84.2%), Random Search (83.5%), PSO (86.7%), GA (87.1%), ACO (86.9%), and standard GOA (87.8%). Additionally, DGOA reduces the average computational time to 120 s and converges in only 30 iterations while achieving the lowest final loss (0.21), highlighting its superior efficiency and convergence stability.
Darwish et al. (Thu,) studied this question.