Fisher-guided personalized federated learning with efficient packed homomorphic aggregation

Existing personalized federated learning (PFL) methods mostly emphasize accuracy while often overlooking privacy leakage under an honest-but-curious server. Moreover, privacy-preserving PFL faces a practical trade-off between utility and the heavy communication overhead induced by homomorphic encryption. To address these challenges, we propose FedFIHE, which leverages Fisher-information-guided adaptive partitioning to dynamically select shared parameters for global aggregation while retaining the remaining parameters for local personalization under heterogeneous data. To make encrypted training efficient, FedFIHE further introduces a packet-level selective encrypted aggregation scheme. Specifically, clients pack model parameters into packets and upload only the top- \(κ \) selected packets with binary masks, and the server performs mask voting to aggregate and broadcast only the most frequently selected packet positions in ciphertext, avoiding the excessive cost caused by inconsistent sparse selections across clients. Extensive experiments on natural image and medical datasets demonstrate that FedFIHE effectively mitigates privacy leakage while achieving better or comparable personalized performance to strong baselines. FedFIHE also remains robust under high sparsity, making it suitable for privacy-sensitive applications such as healthcare.