QLRUN: Integrating Q-learning and covariance matrix learning into Runge–Kutta optimization for engineering design problems

This paper proposes a novel hybrid metaheuristic Q-learning Runge–Kutta Optimization (QLRUN) for solving complex, high-dimensional, rotated, and constrained engineering optimization problems. The proposed framework integrates the gradient-guided search mechanism of the Runge–Kutta Optimization (RUN) algorithm with the adaptive decision-making capability of Q-learning to achieve dynamic regulation of exploration and exploitation behaviors during the optimization process. In QLRUN, the optimization state is characterized using convergence and population-diversity indicators, while the Q-learning agent adaptively selects search operators according to a fitness-improvement-driven reward policy that is iteratively updated throughout the search process. To further enhance rotational robustness and maintain population diversity, a lightweight Covariance Matrix Learning (CML) mechanism is incorporated to construct adaptive eigen-coordinate systems aligned with the fitness landscape's geometry. Unlike the full covariance adaptation employed in CMA-ES, the proposed CML strategy acts as an efficient coordinate-transformation mechanism with low computational overhead. The effectiveness of QLRUN is comprehensively validated using the 23 classic standards, the CEC2020, and CEC2022 benchmark suites, which include unimodal, multimodal, hybrid, composition, shifted, and rotated functions, as well as seven constrained engineering design problems. Comparative evaluations against the original RUN and nine state-of-the-art metaheuristics demonstrate that QLRUN consistently achieves superior convergence accuracy, improved solution stability, and faster convergence characteristics. Quantitatively, QLRUN achieves the top Friedman mean rank across all three benchmark suites, attaining scores of 1.70 on 23 classic standards, 1.40 on CEC2020, and 1.17 on CEC2022, thereby substantially outperforming the original RUN and the competing optimization methods. Moreover, on constrained engineering design problems, QLRUN obtains highly competitive best-known solutions, including a pressure vessel design cost of 5.88536×103 and a tension/compression spring weight of 1.26660×10−2. Beyond p-value-based significance, practical superiority is further confirmed through consistent top-one Friedman rankings and Wilcoxon Rank-Sum win/tie/loss results against RUN of 17/5/1, 8/1/1, and 9/2/1 for the 23 classic standards, CEC2020, and CEC2022 benchmark suites, respectively. Statistical analyses using the Wilcoxon Rank-Sum, Friedman, ANOVA, and Kruskal–Wallis tests confirm the robustness and practical significance of the proposed approach. The results demonstrate that QLRUN constitutes a robust, computationally efficient optimization framework suitable for challenging applications in power systems, energy engineering, and multidisciplinary design optimization.