Multi-objective workload-aware scheduling of reliable and energy-efficient UAV swarms: A multi-agent DQN–PPO framework

Unmanned aerial vehicle (UAV) swarms are increasingly employed in mission-critical operations, where reliable operation and energy-efficient resource utilization are essential. Under dynamic task demands, uneven workload distribution can lead to imbalanced energy consumption and inefficient resource utilization across the UAV swarm. This motivates the need for workload-aware scheduling that dynamically adapts UAV activation and workload distribution according to task demand. To address this challenge, this paper proposes a workload-aware UAV swarm scheduling framework with Activation–Deactivation (AD) capability based on a k -out-of- n :G configuration. The framework jointly models workload dynamics, workload-dependent power consumption, stochastic failures, repairs, and AD operations, thereby capturing the coupled reliability, workload, and energy characteristics of UAV swarm operation. A mathematical model is first developed to characterize the swarm dynamics and is subsequently formulated as a Markov Decision Process (MDP) for learning-based scheduling. The standalone Deep Q-Network (DQN) and Proximal Policy Optimization (PPO) controllers are developed as baseline schedulers under AD and workload-aware AD (WAAD) settings. A cooperative heterogeneous Multi-Agent DQN–PPO (MA–DQN–PPO) framework is then proposed, in which a DQN agent performs discrete UAV–AD decisions, while a PPO agent jointly optimizes continuous workload-balancing intensity and power scaling among active UAVs. In addition, a critical-state-aware reward mechanism is designed to emphasize critical system states, with the same resulting reward provided to both agents to coordinate their heterogeneous decisions toward common scheduling objectives. Accordingly, comprehensive evaluation against the baseline and ablation models, including a Hybrid-PPO controller, demonstrates that the proposed framework achieves an overall reward of 79.6336 and a Multi-Objective Score (MOS) of 0.9681, while maintaining 99.80% availability, 61.83% energy gain, normalized throughput of 3.2568, 325.6834 completed tasks, and a Jain’s fairness index of 0.9997. Although some configurations achieve higher values for individual metrics, the proposed framework achieves the best or comparable performance across most objectives, resulting in the highest MOS and a more balanced overall trade-off. The findings demonstrate that the proposed framework effectively coordinates reliability, energy consumption, and workload distribution for UAV swarm scheduling under dynamic operating conditions.