Federated Learning Enable Training of Perception Model for Autonomous Driving

For intelligent vehicles, a robust perception system relies on training datasets with a large variety of scenes. The architecture of federated learning allows for efficient collaborative model iteration while ensuring privacy and security by leveraging data from multiple parties. However, the local data from different participants is often not independent and identically distributed, significantly affecting the training effectiveness of autonomous driving perception models in the context of federated learning. Unlike the well-studied issues of label distribution discrepancies in previous work, we focus on the challenges posed by scene heterogeneity in the context of federated learning for intelligent vehicles and the inadequacy of a single scene for training multi-task perception models. In this paper, we propose a federated learning-based perception model training system. Through visual explanation, we delve into the relationship between model convergence direction and the content of local data scenes. We also investigate the complex relationships between different perception tasks and the diverse scenarios encountered by vehicles. Subsequently, by utilizing significance detection, the system identifies scene distribution characteristics in different client-local datasets while strategically forming alliances among different vehicle clients. The system effectively balances the scene heterogeneity in different client data and mitigates the performance degradation caused by the inadequacy of a single scene to provide sufficient information for training multiple tasks simultaneously. In our experiments, the system not only outperforms the traditional federated averaging but also demonstrates performance improvements compared to other federated aggregation method.