Joint Service Deployment and Task Scheduling for Satellite Edge Computing: A Two-Timescale Hierarchical Approach

In this paper, we establish a two-timescale framework for the joint service deployment and task scheduling problem in satellite edge computing networks.We aim to optimize the computing performance of networks with diverse quality-of-service (QoS) guarantees for computing tasks. Specifically, to capture the small-timescale network dynamics and task randomness, we formulate the task scheduling problem as a constrained Markov decision process (CMDP) to minimize the energy consumption, load imbalance and packet loss of networks while ensuring the long-term delay. The Lyapunov technique is employed to deal with the delay constraints. A soft actor-critic (SAC)-based deep reinforcement learning (DRL) framework is designed to learn the stationary scheduling policy. We further explore the significant impact of deploying diverse services on the performance of task scheduling in satellite edge computing. Considering that frequent deployment of services will incur huge deployment overhead, we optimize the service deployment on a larger timescale. The optimization problem is modeled as an integer programming problem to improve the service capability of networks and reduce service deployment costs. A heuristic-based atomic orbital search (AOS) approach is proposed to obtain the superior policy with low complexity. Due to the correlation between the problems of two timescales, a hierarchical solution is constructed to iteratively find the excellent solution. Finally, extensive simulations are conducted to validate the effectiveness and superiority of the proposed scheme.