Online Trajectory and Radio Resource Optimization of Cache-Enabled UAV Wireless Networks With Content and Energy Recharging

Recently, unmanned aerial vehicle (UAV)-assisted wireless communication technology has been proposed to exploit the favorable propagation property and flexibility of air-to-ground channels to support content-centric caching and enhance wireless network capacity. In this article, we propose an online UAV-assisted wireless caching design via jointly optimizing UAV trajectory, transmission power and caching content scheduling. Specifically, we formulate the joint optimization of online UAV trajectory and caching content delivery as an infinite-horizon ergodic Markov Decision Process (MDP) problem to obtain a QoE-optimal solution based on the concept of request queues in wireless caching networks. By exploiting the fluid approximation approach, we first derive an optimal control policy from an approximated Bellman equation. Based on this, an actor-critic based online reinforcement learning algorithm is proposed to solve the problem. Finally, simulation results are provided to show that the proposed solution can achieve significant gain over the existing baselines.