Machine Learning-Based Resource Allocation in Satellite Networks Supporting Internet of Remote Things

Satellite networks have been regarded as a promising architecture for supporting the Internet of remote things (IoRT) due to their advantages of wide coverage and high communication capacity in remote areas, which further promotes the development of the satellites for IoRT networks (SIoRTNs). The effectiveness of multi-dimensional resource collaboration has significant impacts on the IoRT data downloading performance. However, the environment ’ s dynamics, e.g., channel conditions and solar infeed process, are unknown in practical scenarios, which poses daunting challenges in making efficient utilization of multi-dimensional resources. Motivated by this fact, we model the joint resource scheduling and IoRT data scheduling problem with the aim of maximizing the amount of the IoRT data of the overall network by applying the model-free reinforcement learning framework. To overcome the limitations of traditional reinforcement learning algorithms, we propose several feature functions by investigating the natural attributes of the multi-dimensional resources of the SIoRTNs, and further exploit the concept of function approximation to approximate the expected downloaded IoRT data given the network state. Furthermore, we propose a state-action-reward-state-action (SARSA) based actor-critic reinforcement learning (SACRL) resource allocation strategy to achieve the optimal resource allocation and IoRT data scheduling with casual information at LEO satellites. Simulations validate the convergence property and the effectiveness of the proposed SACRL algorithm in terms of the amount of the downloaded IoRT data. Particularly, we investigate the impact of typical network parameters on network performance to further provide guidance for future SIoRTN system design.