Federated Learning Showdown: The Comparative Analysis of Federated Learning Frameworks

In this position paper, we underscore the critical need for a systematic and structured approach to comparing Federated Learning (FL) frameworks. Given the diversity of FL frameworks currently available, we argue that a comprehensive comparative analysis is essential for two reasons. First, such an analysis can guide decision-makers in identifying the most suitable FL framework for their specific use case by examining the strengths and weaknesses of each framework. Secondly, it can help researchers recognize the potential gaps and shortcomings in existing FL frameworks, thereby directing further research and development in these areas. To illustrate this, we present our preliminary comparative analysis of 14 different FL frameworks, assessing their individual strengths and weaknesses. Our position advocates for a more methodical understanding and selection of FL frameworks, which we believe will substantially benefit both practical applications and future advancements in the field.