Restart of the JEDi: Dynamic Quality with Just Enough Diversity

In complex control tasks, finding high-performing policies often requires discovering and exploiting specific behavioral strategies. While Quality-Diversity (QD) algorithms can uncover these strategies through extensive behavior space exploration, they sacrifice efficiency by improving suboptimal behaviors. Conversely, Evolution Strategies (ES) achieve impressive performance through focused optimization but frequently become trapped in local optima due to limited behavioral exploration. We present Quality with J ust E nough Di versity (JEDi), a new optimization framework that resolves this fundamental tension. JEDi employs a combination of Gaussian Process modeling and parallel Evolution Strategies to intelligently explore behavioral space while maintaining focused optimization. At its core, JEDi learns a probabilistic mapping between behaviors and performance, using this model to identify and target promising behavioral regions that could unlock better solutions. This targeted exploration is achieved through multiple Evolution Strategy emitters that simultaneously optimize toward selected behaviors while maximizing task performance. To further improve JEDi’s exploration capabilities, we introduce its Dy namic variant DyJEDi with an adaptive restart mechanism that dynamically detects and responds to emitter convergence, independently restarting each Evolution Strategy when it stagnates in both behavior and fitness space. This dynamic approach significantly improves exploration efficiency and robustness to local optima. We demonstrate that DyJEDi outperforms both traditional Evolution Strategies and Quality-Diversity approaches across challenging continuous robotics control tasks, achieving higher final performance. Most notably, DyJEDi solves several hard exploration problems where standard ES methods consistently fail.