COMPACT-FD: Federated distillation and model compression with over-the-air aggregation

Over-the-air federated learning (OTA-FL) enables distributed edge devices to collaboratively train a global model without sharing raw data, leveraging the wireless channel’s superposition property for efficient aggregation. However, practical deployments face challenges due to limited bandwidth, energy, and computational resources, as well as wireless channel impairments. This paper presents COMPACT-FD, a distillation-enhanced OTA-FL framework that combines knowledge distillation (KD), post-training magnitude-based pruning, and over-the-air (OTA) aggregation to characterize SNR-aware compression regimes. The approach combines KD for accuracy and stability, pruning for model compression, and OTA aggregation for communication and resource efficiency. We analyze the interplay between compression and wireless channel conditions, deriving SNR-aware operating recommendations to balance accuracy, model size, and resource constraints. Experiments under varying SNR conditions show that COMPACT-FD achieves substantial reductions in communication and computation costs while maintaining robust accuracy and convergence. Our results provide deployment guidelines for edge intelligence in realistic wireless environments, highlighting the benefits of combining KD, pruning, and OTA aggregation in OTA-FL.