Performance Analysis Of An Intrusion Detection System Based On Big Data Analytics And Ensemble Techniques.

Datasets encompass a wide range of network activities and intrusion patterns. The traditional intrusion detection systems (IDS) are struggling to provide all-round protection to the network but unable to analyze the new volumes of data and the velocity of today's networks. This research leverages the capabilities of big data analytics to process and analyze large-scale datasets collected from network traffic logs. Feature engineering and selection techniques were applied to extract relevant features that capture the distinguishing characteristics of normal and intrusive activities. Each model in the ensemble is trained independently using a subset of the data, utilizing their unique algorithms and strengths. The proposed system employs a range of machine learning models including Support Vector Machines (SVM), Decision Trees, Naive Bayes, k-Nearest Neighbors (KNN), Random Forest, Neural Networks, and two ensemble techniques, Bagging Ensemble, and XGBoosting. A comprehensive comparative analysis of these models were conducted to evaluate their performance in detecting intrusions accurately and efficiently. Hence, a comparative analysis was carried out to evaluate the performance of each model individually and as part of the ensemble. Performance metrics such as accuracy, precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC-ROC) are employed to assess the effectiveness of the models in identifying intrusions and minimizing false positives. The research contributes to the field of intrusion detection by providing insights into the performance of different machine learning models when applied to big data analytics and ensemble techniques. The comparative analysis aids in selecting the most effective models for building robust IDS solutions, improving network security, and safeguarding critical information assets against emerging cyber threats.