Security Information Event Management data acquisition and analysis methods with machine learning principles

In the face of increasing global disruptions, the cybersecurity field is confronting rising threats posed by offensive groups and individual hackers. Traditional security measures often fall short in detecting and mitigating these sophisticated attacks, necessitating advanced intrusion detection methods. The goal of our study is to develop robust network intrusion detection methods using machine learning techniques. In addition, we evaluate the effectiveness of various machine learning models in detecting network intrusions. Model performances are optimized through hyperparameter tuning and feature selection. A range of classification and clustering models have been employed. Data from SIEM systems capturing real-time statistics from cloud-hosted Windows virtual machines has been gathered and augmented with web attack logs from CICIDS2017, each comprising approximately fifteen thousand rows. Hyperparameter tuning, data normalization, standardization and feature selection techniques for model optimization have been used in our study. The research showcases the potential of machine learning in enhancing network intrusion detection capabilities. The findings underscore the effectiveness of the Random Forest Classifier (0.97) and highlight the importance of utilizing diverse datasets and advanced optimization techniques. This study offers valuable insights and sets a foundation for future advancements in cybersecurity strategies and intrusion detection systems.