A rule-based label harmonization framework for cross-dataset IoT intrusion detection

This is the authors' abstract. We don't add key points for this paper.

Machine learning (ML) has become a dominant approach for intrusion detection in Internet of Things (IoT) networks, yet most studies evaluate models within a single dataset, overlooking generalization across heterogeneous environments. This work revisits that challenge through a systematic cross-dataset evaluation using three public datasets: CIC-IoT-2023, IoT-23, and IoTID20. When trained and tested within the same dataset, models achieved near-perfect accuracy (average F1 > 0.99); however, performance degraded sharply when models trained on one dataset were tested on a different one (average F1 < 0.50 except for Multi-Layer Perceptron. Combining multiple datasets into a unified training corpus restored high accuracy (average F1 > 0.99) however, these results must be interpreted with caution, as consistent labeling remains essential for reliable cross-dataset evaluation. These findings highlight the fragility of current Intrusion Detection System (IDS) models to domain shifts and the promise of dataset fusion for enhancing generalization. We introduce a reproducible framework that harmonizes heterogeneous datasets through rule-based labeling grounded in Zeek’s standardized flow features and MITRE ATT&CK / CAPEC semantics. This replaces manual relabeling with a systematic, extensible procedure, enabling diverse traffic to be transformed into interoperable annotations. Labels are assigned by behavioral characteristics rather than dataset-specific categories, allowing discovery of previously unlabeled malicious activity. A proof of concept with three smart-home environments demonstrated the framework: models maintained near-perfect performance when trained on individual or unified datasets (average F1 > 0.99). Overall, unified rule-based labeling enables scalable dataset reuse, improves cross-dataset generalization, and fosters interpretable, data-centric ML for IoT intrusion detection.