A Novel Deep Learning-Based Soil Moisture Prediction Model Using Adaptive Group Radial Lasso Regularized Basis Function Networks (AGRL-RBFN) Optimized by Hierarchical Correlated Spider Wasp Optimizer (HCSWO) and Incremental Learning (IL)

Soil moisture serves as a critical factor in the hydrological cycle, affecting plant growth, ecosystem health, and groundwater reserves. Current methods for monitoring and predicting it fail to account for the complexities introduced by climatic variations and other influencing factors, such as the effects of atmospheric interference and data gaps, leading to reduced prediction accuracy. To address these challenges, this study introduces a novel soil moisture prediction model based on remote sensing and deep learning, utilizing the Adaptive Group Radial Lasso Regularized Basis Function Networks (AGRL-RBFN) optimized by the Hierarchical Correlated Spider Wasp Optimizer (HCSWO) and incremental learning (IL) techniques. The proposed method for monitoring soil moisture utilizes hyperspectral and soil moisture data from a 2017 campaign in Karlsruhe, encompassing variables such as datetime, soil moisture percentage, soil temperature, and remote sensing spectral bands. The proposed methodology begins with comprehensive preprocessing of historical remote sensing data to fill gaps, reduce noise, and correct atmospheric disturbances. It then employs a unique seasonal mapping and grouping technique, enhanced by the AdaK-MCC method, to analyze the impact of climatic changes on soil moisture patterns. The model’s innovative feature selection approach, using HCSWO, identifies the most significant predictors, ensuring optimal data input for the AGRL-RBFN model. The model achieves an impressive accuracy of 98.09%, a precision of 98.17%, a recall of 97.24%, and an F1-score of 98.95%, outperforming existing methods. Furthermore, it attains a mean absolute error (MAE) of 0.047 in gap filling and a Dunn Index of 4.897 for clustering. Although successful in many aspects, the study did not investigate the relationship between soil moisture levels and specific crops, which presents an opportunity for future research aimed at enhancing smart agricultural practices. Furthermore, the model can be refined by integrating a wider range of datasets and improving its resilience to extreme weather conditions, thereby providing a reliable tool for climate-responsive agricultural management and water conservation strategies.