A Green DDPG Reinforcement Learning-Based Framework for Content Caching

Caching popular files at the edge of the network near the users can effectively mitigate the pressure on the backhaul link and reduce the user's transmission delay. The effectiveness of content caching highly depends on the hit rate of the request, so the research of content caching strategy is an important issue. Inspired by reinforcement learning to solve complex control problems, this paper studies the content caching problem that maximizing the use of the cache capacity of the base stations to minimize the average transmission delay of the user request and the energy loss of the network. Because the state of the network is unpredictable and its dimension is huge, the DDPG (Deep Deterministic Policy Gradient) algorithm is used to interact with the environment to learn the optimal cache strategy. By separating the action function network and the value function network, it is possible to optimize each other in the iterative process. Our results show that the proposed framework can improve cache hit ratio and reduce network energy loss.