Deep Reinforcement Learning in High-Dimensional and Stochastic Environments A Short Introductory Literature Survey Critical Methodological Review

This paper presents a review of some of the advancements of the past decade in Deep Reinforcement Learning (DRL), focusing on approaches addressing challenges in high-dimensional and stochastic environments. DRL has gained significant traction in tackling complex decision-making tasks, yet it confronts fundamental difficulties in scenarios with vast state and action spaces and unpredictable dynamics. We examine key studies such as the Dota 2 Five project, demonstrating DRL in multi-agent settings and addressing computational and coordination complexities. Several papers present a methodology for developing their DRL method on the Atari 2600 games. The concept of Value Prediction Functions is explored for managing uncertainty, with discussions on limitations in scalability and accuracy. The paper also analyzes frameworks like MuZero and AlphaZero, highlighting their approaches to model-free and model-based learning in environments with unknown dynamics and issues related to computational demands and generalizability. We identify persistent methodological issues, such as overall neglect for future goals, non-reproducible and non-replicable experiments, incongruent evals and tests, shallow theoretical framing of problems and solutions, and more.