Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learning

The ability to recover from a fall is an essential feature for a legged robot navigate in challenging environments robustly. Until today, there has been little progress on this topic. Current solutions mostly build upon (heuristically) predefined trajectories, resulting in unnatural behaviors and considerable effort in engineering system-specific components. In paper, we present an approach based on model-free Deep Reinforcement (RL) to control recovery maneuvers of quadrupedal robots using a behavior-based controller. The controller consists of four neural policies including three behaviors and one behavior selector to them. Each of them is trained individually in simulation and directly on a real system. We experimentally validate our approach on quadrupedal robot ANYmal, which is a dog-sized quadrupedal system with 12 of freedom. With our method, ANYmal manifests dynamic and reactive behaviors to recover from an arbitrary fall configuration within less 5 seconds. We tested the recovery maneuver more than 100 times, and the rate was higher than 97 %.