A Stability-Centric Framework for Lightweight and Explainable Intrusion Detection

Effective intrusion detection for Internet of Things (IoT) environments requires balancing predictive performance, resource efficiency, and interpretability—particularly in real-world deployments where traffic distributions and attack scenarios vary. While many studies report near-perfect detection on benchmark datasets, this often overlooks model stability under distribution shifts. This paper addresses this gap by introducing a stability-focused evaluation of lightweight, explainable intrusion detection models using compact IoT-23 scenarios and a constrained set of 14 connection-level features for interpretability. Four lightweight models—logistic regression, random forest, XGBoost, and LightGBM—are assessed within a unified pipeline. Beyond standard internal validation, we implement a strict cross-scenario evaluation framework featuring a fully unseen malware capture. Our proposed Internal–External Stability Gap (IESG) framework, enhanced with normalized and multi-metric measures, highlights the degradation in consistency between internal and external metrics. Surprisingly, even models with high internal F1 scores (up to 0.9994) may experience considerable drops in external macro-F1 and specificity, exposing weaknesses in conventional evaluation. Experimentally, LightGBM provides the best trade-off between performance and compactness (606 KB) and shows the smallest stability gap for malicious detection. Nevertheless, all models show reduced balanced performance under scenario shift, underscoring that deployment readiness hinges on stability under changing conditions. Feature ablation reveals that leveraging high-impact features, such as port information, can boost internal accuracy at the expense of generalization. In summary, we demonstrate that while lightweight models deliver strong detection, only those proven stable across scenarios are viable for real-world IoT intrusion detection. Our evaluation framework offers a practical, interpretable tool for assessing model robustness.