Abstract. We present a large-scale evaluation of federated learning (FL) in Internet of Things (IoT) environments with up to 5,000 heterogeneous devices. Through 150 controlled experiments, we analyze security vulnerabilities under four attack types (data poisoning, model poisoning, Byzantine, backdoor) and evaluate four defense mechanisms (FedAvg, Krum, Trimmed Mean, Coordinate-wise Median). We report scalability trends (devices vs. accuracy/latency) and per-attack success rates with confidence intervals. Statistical analysis (two-sided tests with effect sizes) supports our findings. Results show that FL accuracy degrades significantly beyond 2,000 devices, attack success rates average 78.2%, and detection rates remain below 31% even with advanced defenses. Krum provides the best balance between accuracy preservation (82.7%) and attack detection (23.2%). The full dataset, seeds, and scripts are available in the supplementary material.
Atmoko et al. (Tue,) studied this question.