Federated Learning with Privacy-Preserving Active Learning: A Min-Max Mutual Information Approach

In an era where data privacy is paramount, Federated Learning (FL) and Active Learning (AL) have emerged as pivotal paradigms for building intelligent systems that respect user confidentiality. This paper introduces a novel FL framework enhanced by AL, orchestrated by the Min-Max Mutual Information (MMMI) principle, and fortified with Differential Privacy (DP) to ensure robust privacy preservation. Our approach adeptly addresses the prevalent challenge of unbalanced data distribution in federated settings. By employing MMMI, the framework adeptly pinpoints and assimilates the most informative samples across distributed datasets, ensuring comprehensive representation even for underrepresented classes or features. The integration of DP within the training process, as delineated in Algorithm 1, serves to maintain strict privacy controls, aligning with the privacy budget constraints and ensuring the confidentiality of the data remains intact during model updates. The local computation on devices preserves data sovereignty, while the MMMI-based AL mechanism judiciously minimizes the labeling requirements. It enhances the global model's performance by selectively incorporating instances yielding the maximum informational benefit. Empirical results from experiments on MNIST and CIFAR datasets underscore the framework's effectiveness in cultivating robust, efficient, and discreet AI models. The research presented here marks a significant advance in the amalgamation of FL, AL, and DP. It sets a robust foundation for future exploration into their convergence, especially for applications demanding stringent privacy considerations.