Federated Reinforcement Learning for Fast Personalization

Understanding user behavior and adapting to it has been an important focus area for applications. That adaptation is commonly called Personalization. Personalization has been sought after in gaming, personal assistants, dialogue managers, and other popular application categories. One of the challenges of personalization methods is the time they take to adapt to the user behavior or reactions. This sometimes is detrimental to user experience. The contribution of this work is twofold: (1) showing the applicability of granular (per user) personalization through the use of reinforcement learning, and (2) proposing a novel mitigation strategy to decrease the personalization time, through federated learning. To our knowledge, this paper is among the first to present an overall architecture for federated reinforcement learning (FRL), which includes the grouping policy, the learning policy, and the federation policy. We demonstrate the efficacy of the proposed architecture on a non-player character in the Atari game Pong, and scale the implementation across 3, 4, and 5 users. We demonstrate the success of the proposal through achieving a median improvement of ~17% on the personalization time.