The increasing heterogeneity of cloud, enterprise, and Internet of Things (IoT) environments raises concerns about the long-term reliability of machine-learning-based intrusion detection systems (IDSs). This study evaluates temporal robustness and cross-domain generalization using four publicly available datasets collected between 2017 and 2024. Five representative models—Random Forest, Gradient Boosting, Multi-Layer Perceptron, Autoencoder, and a lightweight 1D-CNN—are assessed under in-dataset, forward temporal, enterprise-to-IoT transfer, and dataset-agnostic evaluation protocols without retraining. In the dataset evaluation, models achieve Macro-F1 scores between 0.84 and 0.96. However, forward temporal testing reveals consistent degradation, with performance reductions reaching ΔF1 ≈ 0.20–0.27 when models trained on 2017 enterprise traffic are applied to IoT datasets from 2023 to 2024. Under cross-domain transfer, Macro-F1 decreases to 0.69–0.78, and benign false-positive rates increase up to 0.30, indicating substantial sensitivity to traffic distribution shifts. Tree-based ensemble models show comparatively lower degradation (≈6–23%) and reduced performance variance across datasets. Semantic feature analysis further indicates that flow intensity and temporal activity features exhibit higher cross-dataset stability than protocol-dependent indicators. These findings demonstrate that IDS robustness in evolving Internet environments depends strongly on evaluation methodology and feature stability, highlighting the need for generalization-oriented assessment strategies.
Elangovan et al. (2026) studied this question.