What question did this study set out to answer?

The aim is to create a federated learning algorithm that optimizes model training while addressing data variability and adapting to new tasks.

March 1, 2026

A Multi-objective Alternating Knowledge Distillation Method for Heterogeneous Federated Learning

Key Points

The aim is to create a federated learning algorithm that optimizes model training while addressing data variability and adapting to new tasks.
Developed a multi-objective optimization approach for federated learning.
Introduced a federated alternating learning framework with two knowledge distillation modes.
Implemented a multi-objective embedded optimization for sampling clients and decision space.
Selected Pareto solutions for balancing plasticity and stability before communication.
Demonstrated effective mitigation of the data heterogeneity issue in federated learning.
Achieved improved balance between preventing catastrophic forgetting and task adaptation.

Abstract

Federated learning has shown great potential in protecting client privacy during large-scale model training. However, the challenge of non-IID data in Industrial Internet of Things environments makes it difficult to balance between preventing catastrophic forgetting and adapting to new tasks during model training. This paper proposes a Federated Learning algorithm via multi-objective optimization and continual learning (FedMoCL). Specifically, a novel federated alternating learning (FedAL) Framework is designed, which alternately optimizes plasticity and stability in the Continual Learning process through two distinct knowledge distillation modes: Teacher–Student Knowledge Distillation from the current server to the current client and self–knowledge distillation from historical clients to the current client. Furthermore, a multi-objective embedded optimization (MEO) is designed for FedAL, where sampled clients are considered as the evolutionary population and the entire set of clients as the decision space. Before federated communication, the plasticity-stability trade-off Pareto Solution is selected for aggregation. Simulation results demonstrate that FedMoCL effectively alleviates the data heterogeneity problem in federated foundation model training.

AI से पूछें

Bookmark