Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks

Multi-agent deep reinforcement learning (MARL) suffers from a lack of-used evaluation tasks and criteria, making comparisons between difficult. In this work, we provide a systematic evaluation and of three different classes of MARL algorithms (independent learning, multi-agent policy gradient, value decomposition) in a diverse of cooperative multi-agent learning tasks. Our experiments serve as a for the expected performance of algorithms across different learning, and we provide insights regarding the effectiveness of different approaches. We open-source EPyMARL, which extends the PyMARL codebase include additional algorithms and allow for flexible configuration of implementation details such as parameter sharing. Finally, we-source two environments for multi-agent research which focus on under sparse rewards.