Sparse and intelligent command of unmanned swarms via hierarchical multiagent reinforcement learning

The integration of unmanned swarms into manned/unmanned cooperative systems in modern defence operations is severely constrained by commander decision-making overload and communication links, creating a critical command and control (C2) challenge. To this end, this paper proposes HSI-HMARL, a novel hierarchical multiagent reinforcement learning (HMARL) framework specifically designed for sparse commands in human-swarm interaction (HSI), consisting of a novel C2 paradigm together with the corresponding learning approach for intelligent decision-making. By fusing the system-level C2 hierarchy with the policy-level algorithmic hierarchy, HSI-HMARL allows a command agent to direct the entire swarm as a single abstract agent by selecting from a library of interpretable, pre-trained tactical behaviors, i.e., joint macro-actions. This approach drastically reduces communication bandwidth requirements and lowers the decision-making load of the command agent, enabling effective manned/unmanned cooperation through intuitive, high-level intervention. This approach makes command complexity independent of swarm size, reducing communication bandwidth and the cognitive load of the commander. Both simulations and real-world robot experiments demonstrate that the proposed approach achieves efficient swarm control characterized by high learning efficacy, reduced communication overhead, and seamless human intervention, indicating the potential for real-world defence applications.