Coordination of Exploration and Exploitation in a Dynamic Environment

One much researched issue in reinforcement learning is the trade-off between exploration and exploitation. Being able to effectively balance exploration and exploitation activities becomes even more crucial in a dynamic environment. A learning algorithm is proposed herein that provides one solution to the exploration vs. exploitation dilemma. The algorithm is presented in the context of a path-finding agent in a dynamic grid-world problem. The state-value function used is penalty-based, allowing the agent to act over the space of paths with minimal penalties. A forgetting mechanism is implemented that allows the agent to explore paths that were previously determined to be suboptimal. Simulation results are used to analyze the behavior of the proposed algorithm in a dynamic environment.