In multi-agent tasks, the credit assignment problem has garnered widespread attention, as an effective credit assignment mechanism can not only promote cooperation among agents but also significantly enhance the efficiency and quality of task completion. Reasonable credit assignment ensures that each agent receives its due rewards, thus motivating them to participate and coordinate more actively, which is crucial for successfully completing complex tasks. To address this issue, we propose an improved algorithm based on the MAPPO algorithm. This algorithm employs a centralized training and decentralized execution framework, incorporates a counterfactual baseline, and uses an enhanced action value network as a centralized critic. By fixing the behaviors of other agents and marginalizing the actions of a single agent, we can more accurately assess each agent's contribution to the overall team performance. This approach not only ensures the fairness of credit assignment but also allows for a granular assessment of each agent's specific performance, achieving a more refined reward mechanism. To validate the effectiveness of our improved algorithm, we conducted experiments in a collaborative robotic navigation task. The results indicate that our method outperforms existing approaches in overall performance.
Qian et al. (Mon,) studied this question.