Clustered Federated Learning With Adaptive Local Differential Privacy on Heterogeneous IoT Data

The Internet of Things (IoT) is penetrating many aspects of our daily life with the proliferation of artificial intelligence applications. Federated learning (FL) has emerged as a promising paradigm enabling many intelligent IoT applications; however, the transmitted model gradients or weights still encode private information, which can be exploited to launch inference attacks. One popular way is to apply local differential privacy (LDP) into FL. However, existing work does not provide a practical solution due to two issues. First, the fine-grained range difference of weights in different layers of an FL model has not been explicitly considered. Second, the accumulated privacy budget may cause a budget explosion. In this article, we propose a local differentially private scheme to train clustered FL models on heterogeneous IoT data by using adaptive clipping, weight compression, and parameter shuffling (namely, ACS-FL), aimed at mitigating the curse of dimensionality, the amount of LDP noise, and the communication overhead of IoT devices. Empirical evaluations on MNIST, fashion-MNIST, and Federated Extended MNIST demonstrate that ACS-FL achieves a superior performance in balancing the tradeoff between privacy and utility.