Reinforcement Learning Based Capacity Management in Multi-Layer Satellite Networks

The development of satellite networks is drawing much more attention in recent years due to the wide coverage ability. Composed of geosynchronous orbit (GEO), medium earth orbit (MEO), and low earth orbit (LEO) satellites, the satellite network is a three-layer heterogeneous network of high complexity, for which comprehensive theoretical analysis is still missing. In this paper, we investigate the problem of capacity management in the three-layer heterogeneous satellite network. We first construct the model of the network and propose a low-complexity method for calculating the capacity between satellites. Based on the time structure of the time expanded graph, the searching space is greatly reduced compared to traditional augmenting path searching strategies, which can significantly reduce the computing complexity. Then, based on Q-learning, we proposed a long-term optimal capacity allocation algorithm to optimize the long-term utility of the system. In order to reduce the storage and computing complexity, a learning framework with low-complexity is constructed while taking the properties of satellite systems into account. Finally, we analyze the capacity performance of the three-layer heterogeneous satellite network and also evaluate the proposed algorithms with numerical results.