Research on the evolution mechanism of multi-agent reinforcement learning marl simulation volleyball cooperative tactics

In a dynamic adversarial environment, the automatic evolution of multi-agent collaborative strategies is a frontier topic in reinforcement learning research. In this study, a simplified two-dimensional volleyball game is used as a simulation environment to explore the internal mechanism of agent autonomous evolution of collaborative tactics under the Multi-Agent Reinforcement Learning (MARL) framework based on Centralized Training with Decentralized Execution (CTDE). The experiment adopts the self-game training mode and designs an evaluation system including three indices: offensive efficiency, defensive success rate, and formation coordination degree. After more than 1 million rounds of training, the average score rate of the agent team in the test set reached 72.3%, which was 58.1% higher than the random baseline strategy. The entropy value of the team decreased from the initial 5.01 to 1.82 in the stable period, which indicated that its tactical synergy was significantly enhanced, and the basic tactical modes such as rotation compensation and cross cover were spontaneously formed. The innovations of this research are mainly reflected in three aspects: First, a simplified volleyball simulation environment with continuous state and discrete action space is constructed, which provides a benchmark for studying discrete–continuous mixed decision-making problems; Secondly, an improved credit allocation mechanism based on local observation is designed, which effectively alleviates the problem of ‘‘lazy agent’’ in multi-agent collaboration; Third, by introducing opponent modeling and strategy distillation technology, the evolution process of adversarial tactics is accelerated and the robustness of strategies is improved. The results provide an empirical basis for understanding the emergence mechanism of multi-agent collaborative strategies in open environments.