Evolutionary Strategies (ESs) are optimization metaheuristics largely adopted in Evolutionary Computation (EC). Since their introduction in the early 70s, researchers in the field have attempted to improve the efficacy of these algorithms. The most advanced ESs, such as the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) and Exponential Natural Evolution Strategies (xNESs), make use of covariance matrices storing relationships between parameters to be optimized, which enable the algorithms to fasten the search in the solution spaces. However, the computational cost of calculating covariance matrices linearly scales with the number of parameters. Recently, the OpenAI Evolutionary Strategy (OpenAI-ES) emerged as an effective ES in different domains, thanks to the parameter information stored in two momentum vectors. Furthermore, OpenAI-ES gains an advantage from the usage of symmetric sampling and weight decay techniques. In this work, I delve into the application of symmetric sampling and weight decay on CMA-ES, xNES and Separable Natural Evolution Strategies (sNESs), with the aim to improve their performance in domains in which they get stuck in local minima outcomes. Specifically, I propose three novel variants for each ES and verify their efficacy with respect to the PyBullet halfcheetah and hopper robot locomotion problems, and two collective tasks (i.e., swarm aggregation and swarm foraging). The findings reveal that symmetric sampling produces performance enhancements in all the domains, whereas the effect of weight decay varies across the considered problems. Furthermore, symmetric sampling allows ESs to keep parameter size limited, which is paramount in these scenarios. This research identifies techniques enhancing the success of modern ESs, proposes several ES variants, and discusses the relationship between algorithmic performance and task properties.
Paolo Pagliuca (Tue,) studied this question.