Hierarchical reinforcement learning with opponent modeling for command and control system

Abstract This study focuses on the challenges faced by deep reinforcement learning (DRL) methods in intelligent decision-making for air defense command and control (C2) systems, such as the dimensional catastrophe of the state and action space and the lack of assurance of convergence. Aiming at these problems, this paper proposes a novel large-scale oriented hierarchical architecture for air defense decision-making based on the hierarchical mechanism (HM), which decomposes the air defense decision-making problem into the task allocation problem in the upper layer and the task execution problem in the lower layer. Firstly, the operational environment is divided into multiple zones, and combat units in each combat zone are abstracted into multiple executor agents, on the basis of which a commander agent is abstracted in each combat zone; secondly, a hierarchical collaborative decision-making task allocation algorithm based on TAOM-MAAC is proposed and the agents at each level are trained. Finally, the algorithm proposed is simulated and verified in the digital simulation environment, which proves the rationality and effectiveness of the method.