Simple random search provides a competitive approach to reinforcement learning

A common belief in model-free reinforcement learning is that methods based on search in the parameter space of policies exhibit significantly worse complexity than those that explore the space of actions. We dispel such by introducing a random search method for training static, linear for continuous control problems, matching state-of-the-art sample on the benchmark MuJoCo locomotion tasks. Our method also finds a optimal controller for a challenging instance of the Linear Quadratic, a classical problem in control theory, when the dynamics are not. Computationally, our random search algorithm is at least 15 times more than the fastest competing model-free methods on these benchmarks. We advantage of this computational efficiency to evaluate the performance of method over hundreds of random seeds and many different hyperparameter for each benchmark task. Our simulations highlight a high in performance in these benchmark tasks, suggesting that commonly estimations of sample efficiency do not adequately evaluate the of RL algorithms.