Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

We investigate the optimization aspects of personalized Federated Learning(FL). We propose general optimizers that can be applied to numerous existing FL objectives, specifically a tailored variant of Local SGD and of accelerated coordinate descent/accelerated SVRCD. By examining a personalized objective capable of recovering many existing personalized objectives as special cases, we develop a comprehensive optimization theory to a wide range of strongly convex personalized FL models in the. We showcase the practicality and/or optimality of our methods in of communication and local computation. Remarkably, our general solvers and theory can recover the best-known communication and guarantees for addressing specific personalized FL objectives., our proposed methods can serve as universal optimizers, rendering design of task-specific optimizers unnecessary in many instances.