Federated deep reinforcement learning for privacy-preserving offloading in vehicular edge computing

Abstract With the rapid development of Internet of Vehicles (IoV) applications, the demand for serving computation-intensive and delay-sensitive tasks, which are executed in a dynamic mobility environment, continues to grow, while the embedded computing power carried by vehicles remains limited, and they are facing strict requirements in terms of latency. In vehicular edge computing, to offload computation to the nearby roadside units (RSUs) and to enable centralized learning-based offloading, mobility, task, channel, and resource information at the whole system needs to be gathered at a central learner, leading to a higher communication overhead and raw data exposure. This study introduces a privacy-aware federated deep reinforcement learning (FDRL) framework for vehicular edge computing task offloading with RSU assistance. The novelty of the proposed framework does not lie in the common usage of federated learning and deep reinforcement learning (DRL), but rather in the compactness of four coupled mechanisms: generation of a hybrid action representation of federated binary offloading decision and continuous resource allocation for RSUs, a personalized federated aggregation mechanism for non-IID vehicular observations collected on the RSU, a task-criticality-aware deadline reliability model with class-dependent violation penalties, and a handover-aware multi-RSU model that incorporates signaling delay, service-context transfer delay, and processing/authentication delay. In the proposed framework, the model parameters of the local SAC policies are provided to the federated coordinator rather than the locally observed information, such as raw vehicular trajectories, which can instead be used for local training of the SAC-based policies. Controlled simulation experiments are conducted to compare the proposed method with both local execution and edge-offloading methods, two centralized DRL baselines (DQN and DDPG), and three federated DRL baselines (FedAvg-DQN, centralized-SAC, and federated-MADRL). The results indicate that under the adopted simulation settings, the proposed FDRL framework achieves competitive and/or better system cost, delay, energy, deadline-violation performance, and communication overhead compared with other schemes. This privacy usefulness really means having less raw data exposed when federated training is used, and does not mean any formal privacy guarantee against inference attacks against model updates.