Network intrusion detection system for IoT security using machine learning and statistical based hybrid feature selection

Abstract The widespread adoption of Internet of Things (IoT) devices has revolutionized daily life, offering convenience and efficiency. However, this growth has also brought new security challenges. With the escalating use of Internet and network technology, the number of cyber‐attacks has increased, intensifying the focus on Intrusion Detection Systems (IDS) among researchers. Network intrusion detection (NID) plays a crucial role in securing IoT networks, becoming essential for modern security infrastructure. Recently, machine learning algorithms have shown promise in providing IDS solutions. Yet, IoT IDS systems face challenges due to their functional and physical diversity, making comprehensive feature utilization impractical. Therefore, effective feature selection becomes imperative. In this research, a novel feature selection methodology for anomaly‐based NIDS is proposed. The methodology commences by employing two filter‐based techniques, namely 1‐way ANOVA and the Pearson correlation coefficient, to meticulously identify and extract pertinent features from the dataset. These methods serve as initial filters to discern the most relevant attributes, ensuring that only the most informative features are retained for subsequent analysis. Subsequently, the most optimal features identified by both methodologies are extracted utilizing the principles of union and intersection in mathematical set theory. Using the NSL‐KDD and UNSW‐NB15 datasets, we exemplify how our model can outperform conventional ML classifiers in terms of detection rate, precision, recall. In our study, intrusion detection is carried out by SVM, kNN, Decision Tree, Logistic Regression and Random Forest using trained attack patterns. The demonstrated results highlight the exceptional performance of the proposed system, with an impressive accuracy rate of 99.6% on the NSL‐KDD dataset and a substantial 97.7% on the UNSW‐NB15 dataset, clearly surpassing the performance of contemporary methods.