Deep ExRL: Experience-Driven Deep Reinforcement Learning in Control Problems

Reinforcement Learning (RL) has demonstrated significant accomplishments in various fields but struggles with adaptability. A common problem emerges when an agent trained to ascend a hill fails to maintain its performance under changes in conditions such as gravity or slope. Our study focuses on on-policy deep RL algorithms and enhances them by allowing the abstraction of experiences to be transferred to new agents. This approach facilitates accelerated learning through improved exploration and reduces the need for exhaustive hyperparameter optimization. This transfer can occur from the same environment or a similar one. We focus on a simple control problem as a case study. The results show a 50% increase in training success rate and a 15% boost in sample efficiency.