In multi-agent reinforcement learning (MARL), the large number of agents can lead to information overload, hindering effective learning in large-scale systems such as industrial control and smart manufacturing environments. While mean field methods offer scalable solutions in homogeneous agent environments, real-world scenarios often involve heterogeneous agent decision making and dynamic interaction structures. To address these challenges, attention-based adaptive mean field methods have emerged. However, they still face limitations: (1) neighbor-weighted mean field lacks a global perspective, limiting the ability to model global coordination in systems; (2) single-scale mean field representations struggle to capture multi-level agent interactions critical for scene interaction optimization. To overcome these limitations, we propose a multi-scale mean field reinforcement learning framework, integrating far-field global action distributions with near-field local interactions weighted by attention. By leveraging multi-head attention, our method comprehensively captures interactions at different scales, enabling more adaptive decision making. Experimental results demonstrate superior performance and scalability across various benchmark tasks, highlighting its potential for enhancing decision-making in complex environments.
Li et al. (Thu,) studied this question.