A Comparative Study of PSO and CMA-ES Algorithms on Black-box Optimization Benchmarks

Numerous practical engineering applications can be formulated as non-convex, non-smooth, multi-modal and ill-conditioned optimization problems. Classical, deterministic algorithms require an enormous computational effort, which tends to fail as the problem size and its complexity increase, which is often the case. On the other hand, stochastic, biologically-inspired techniques, designed for global optimum calculation, frequently prove successful when applied to real life computational problems. While the area of bio-inspired algorithms (BIAs) is still relatively young, it is undergoing continuous, rapid development. Selection and tuning of the appropriate optimization solver for a particular task can be challenging and requires expert knowledge of the methods to be considered. Comparing the performance of viable candidates against a defined test bed environment can help in solving such dilemmas. This paper presents the benchmark results of two biologically inspired algorithms: covariance matrix adaptation evolution strategy (CMA-ES) and two variants of particle swarm optimization (PSO). COCO (COmparing Continuous Optimizers) – a platform for systematic and sound comparisons of real-parameter global optimization solvers was used to evaluate the performance of CMA-ES and PSO methods. Particular attention was paid to the efficiency and scalability of both techniques.