Multi-Agent DRL-Based Coded Caching and Resource Allocation in UAV-Assisted Networks

In emergency communications constrained by bandwidth limitations, unmanned aerial vehicle (UAV)-based coded caching presents a promising approach for the efficient dissemination of high-bandwidth-demanding services. This paper focuses on content download and content repair in aerial caching networks, where UAVs deliver contents to both ground users and invalid UAVs. To address potential data loss due to limited power and high mobility, fault-tolerant codes are utilized to maintain data availability and reliability. Initially, we derive the expressions of communication cost and success rate for content download and content repair. The size of coded fragments, determined by the coding design, affects both the success rate and transmission cost, while the resource allocation, which influences the cooperative relationships, also impacts these two aspects. The interplay between coding design and resource allocation is thus established to jointly optimize the overall performance. Then, we design a joint optimization problem of erasure coding schemes, coding parameters, matching relations, and UAV trajectories to maximize the overall success rate. Moreover, we propose a hierarchical multi-agent parameterized deep Q-network (H-MA-PDQN) algorithm integrating a dual-component structure for long-term coding and immediate resource allocation to solve the mixed integer nonlinear programming (MINLP), and each agent employs a PDQN with hybrid discrete-continuous action space. Simulation results demonstrate that our proposed H-MA-PDQN algorithm increases the success probability by 26.7% and 66.7% and reduces the transmission cost by 27.3% and 42.9% compared with the DQN and greedy-based strategies, respectively.