Privacy-Preserving Intelligent Resource Allocation for Federated Edge Learning in Quantum Internet

Federated learning (FL) is an emerging technology for empowering various applications that generate large amounts of data in intelligent cyber-physical systems (ICPS). Though FL can address users' concerns about data privacy, its maintenance still depends on efficient incentive mechanisms. For long-term incentivization to participants in data federation under dynamic environments, deep reinforcement learning as a promising technology has been extensively studied. However, the non-stationary problem caused by the heterogeneity of ICPS devices results in a serious effect on the convergence rate of existing single-agent reinforcement learning. In this paper, we propose a multi-agent learning-based incentive mechanism to capture the stationarity approximation in FL with heterogeneous ICPS. First, we formulate the secure communication and data resource allocation problem as a Stackelberg game in FL with multiple participants. Then, to tackle the heterogeneous problem, we model this multi-agent game as a partially observable Markov decision process. Particularly, a multi-agent federated reinforcement learning algorithm is proposed to learn the allocation policies efficiently by dwindling variances in policy evaluation caused by interaction among multiple devices without sharing privacy information. Moreover, the proposed algorithm is proved to attain convergence at an expected rate. Lastly, extensive experimental results demonstrate that our proposed algorithm significantly outperforms baselines.