DRL-Based Secure Aggregation and Resource Orchestration in MEC-Enabled Hierarchical Federated Learning

Federated learning (FL) provides a new paradigm for protecting data privacy by enabling model training at devices and model aggregation at servers. However, data information may be leaked to honest-but-curious aggregation servers by updated model parameters. The existing secure methods do not fully exploit the potentiality of data characteristics in enhancing security, which makes it impossible to optimize limited system resources overall to achieve secure, fair, and efficient FL systems. In this article, a DRL-based joint secure aggregation and resource orchestration scheme is proposed to guarantee security and fairness, and improve efficiency for hierarchical FL (HFL) assisted by untrusted mobile-edge computing (MEC) servers. We formulate a joint optimization problem of data size, payment, and resource orchestration, to maximize the long-term social welfare subject to secure aggregation and limited resources. Since the formulated problem is a complex mixed integer dynamic optimization problem with NP-hardness, where multiple mixed integer optimization variables are highly coupled in time-varying constraints and objective function, it is difficult to obtain its optimal solution via traditional optimization methods. Thus, we propose a hierarchical reward function-based DRL algorithm (MATD3) to guide the agents to approach the optimal policy of secure aggregation and resource orchestration. Simulation results show that the proposed algorithm MATD3 can achieve superior performance over comparison algorithms and the MEC-enabled HFL framework outperforms two-layer FL frameworks.