Optimizing Machine Learning-Based Intrusion Detection Systems: A Comparative Study on KDD99 and BKIDS2025 Datasets

The increasing sophistication and frequency of cyberattacks make intrusion detection systems (IDS) essential, yet many machine learning–based IDS studies still rely on legacy benchmarks such as KDD99, raising concerns about real-world generalization. This study optimizes and comparatively evaluates supervised IDS models on KDD99 and BKIDS2025, a contemporary dataset collected from a production university network (1,463,847 labeled flows over 30 days) covering modern protocols and usage patterns, including TLS 1.3, DNS-over-HTTPS, and IoT traffic. Five supervised models—Logistic Regression, Random Forest, XGBoost, Voting, and Stacking—are assessed under a unified pipeline using precision, recall, F1-score, and false alarm rate. Ensemble methods consistently outperform individual classifiers across both datasets. The Stacking Classifier achieves the best results, reaching 99.8% recall on KDD99 and 96.0% recall with a 97.2% F1-score on BKIDS2025. A Wilcoxon signed-rank test supports the statistical significance of the ensemble advantage, and runtime profiling indicates sub-millisecond inference latency, enabling real-time deployment. The performance drop on BKIDS2025 highlights the limitations of benchmark-only evaluation and motivates routine validation on up-to-date, real-world datasets for robust IDS design.