Nonconvex optimization presents significant challenges in many fields, particularly in training deep neural networks (DNNs), where poor local minima can degrade generalization-especially with limited data. High-dimensional nonconvex optimization presents two central challenges: 1) effectively balancing global exploration with rapid local exploitation and 2) establishing convergence guarantees, particularly with sparse individuals under nonsmooth regularizations. To address these limitations, we propose adaptive niching-based gradient-accelerated DE (AdaptiveGDE), an AdaptiveGDE differential evolution (DE) algorithm. It introduces a novel two-step mutation operator that decouples differential mutation and gradient descent, allowing independent control of exploration and exploitation. An adaptive niching strategy dynamically adjusts the number of subpopulations based on population similarity and iteration progress, enabling diverse early exploration and refined late-stage convergence. Under relaxed smoothness assumptions and approximate ₁ regularization, we provide convergence guarantees in expectation to a near-optimal solution within O (1/ ^4) iterations. Extensive experiments show that AdaptiveGDE achieves robust global exploration on complex multimodal functions, strong local exploitation on convex problems, and significantly improves test accuracy and loss in DNN training, especially under limited data scenarios.
Yu et al. (Thu,) studied this question.