Cross-domain resources optimization for hybrid edge computing networks: federated DRL approach

Due to the dynamic nature of service requests and the uneven distribution of services in the Internet of Vehicles (IoV), Multi-access Edge Computing (MEC) networks with pre-installed servers are often susceptible to insufficient computing power at certain times or in certain areas. In addition, Vehicular Users (VUs) need to share their observations for centralized neural network training, resulting in additional communication overhead. In this paper, we present a hybrid MEC server architecture, where fixed RoadSide Units (RSUs) and Mobile Edge Servers (MESs) cooperate to provide computation offloading services to VUs. We propose a distributed federated learning and Deep Reinforcement Learning (DRL) based algorithm, namely Federated Dueling Double Deep Q-Network (FD3QN), with the objective of minimizing the weighted sum of service latency and energy consumption. Horizontal federated learning is incorporated into the Dueling Double Deep Q-Network (D3QN) to allocate cross-domain resources after the offload decision process. A client-server framework with federated aggregation is used to maintain the global model. The proposed FD3QN algorithm can jointly optimize power, sub-band, and computational resources. Simulation results show that the proposed algorithm outperforms baselines in terms of system cost and exhibits better robustness in uncertain IoV environments.