Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning

Although reinforcement learning methods can achieve impressive results in, the real world presents two major challenges: generating samples is expensive, and unexpected perturbations or unseen situations cause but specialized policies to fail at test time. Given that it is to train separate policies to accommodate all situations the agent see in the real world, this work proposes to learn how to quickly and adapt online to new tasks. To enable sample-efficient learning, we learning online adaptation in the context of model-based reinforcement. Our approach uses meta-learning to train a dynamics model prior such, when combined with recent data, this prior can be rapidly adapted to the context. Our experiments demonstrate online adaptation for continuous tasks on both simulated and real-world agents. We first show simulated adapting their behavior online to novel terrains, crippled body parts, highly-dynamic environments. We also illustrate the importance of online adaptation into autonomous agents that operate in the real by applying our method to a real dynamic legged millirobot. We the agent's learned ability to quickly adapt online to a missing, adjust to novel terrains and slopes, account for miscalibration or errors pose estimation, and compensate for pulling payloads.