Byzantine-Robust Multimodal Federated Learning Framework for Intelligent Connected Vehicle

In the rapidly advancing domain of Intelligent Connected Vehicles (ICVs), multimodal Federated Learning (FL) presents a powerful methodology to harness diverse data sources, such as sensors, cameras, and Vehicle-to-Everything (V2X) communications, without compromising data privacy. Despite its potential, the presence of Byzantine adversaries–malicious participants who contribute incorrect or misleading updates–poses a significant challenge to the robustness and reliability of the FL process. This paper proposes a Byzantine-robust multimodal FL framework specifically designed for ICVs. Our framework integrates a robust aggregation mechanism to mitigate the influence of adversarial updates, a multimodal fusion strategy to effectively manage and combine heterogeneous input data, and a global optimization objective that accommodates the presence of Byzantine clients. The theoretical foundation of the framework is established through formal definitions and equations, demonstrating its ability to maintain reliable and accurate learning outcomes despite adversarial disruptions. Extensive experiments highlight the framework’s efficacy in preserving model performance and resilience in real-world ICV environments.