Graph-Assisted On-Policy Reinforcement Learning for Fair Multi-UAV Trajectory Optimization in IoT Networks

Unmanned aerial vehicles (UAVs) serving as mobile base stations (UAV-BSs) offer a flexible solution for providing on-demand wireless connectivity to Internet of Things (IoT) devices. However, optimizing their trajectories to maximize communication performance in dynamic and complex environments remains a significant challenge. Traditional optimization and learning-based methods often struggle to ensure stable coordination and scalable performance in such non-stationary, decentralized multi-agent environments.This paper proposes GNN-SARSA, a graph-assisted on-policy decentralized reinforcement learning framework in which Graph Neural Network (GNN)-based relational state aggregation is combined with State-Action-Reward-State-Action (SARSA) updates for fair multi-UAV trajectory optimization. The GNN is employed to effectively capture the complex spatial-topological relationships between UAVs and ground-based IoT devices, enabling more informed decision-making, while SARSA enables stable policy refinement over a discretized interaction space. Our approach is a meticulously designed reward and penalty mechanism that balances the dual objectives of maximizing overall system throughput and ensuring service fairness among all IoT devices, quantified using Jain’s Fairness Index. Through extensive simulations, we evaluate the proposed GNN-SARSA method in both single- and multi-UAV scenarios, considering varying IoT device densities and communication ranges.The simulation results show that the proposed framework improves fairness and learning stability compared with the MLP-SARSA baseline and achieves competitive performance relative to representative MARL baselines, including Proximal Policy Optimization, QMIX, and Independent Q-Learning.