Constructing important features from massive network traffic for lightweight intrusion detection

Efficiently processing massive data is a big issue in high‐speed network intrusion detection, as network traffic has become increasingly large and complex. In this work, instead of constructing a large number of features from massive network traffic, the authors aim to select the most important features and use them to detect intrusions in a fast and effective manner. The authors first employed several techniques, that is, information gain (IG), wrapper with Bayesian networks (BN) and Decision trees (C4.5), to select important subsets of features for network intrusion detection based on KDD'99 data. The authors then validate the feature selection schemes in a real network test bed to detect distributed denial‐of‐service attacks. The feature selection schemes are extensively evaluated based on the two data sets. The empirical results demonstrate that with only the most important 10 features selected from all the original 41 features, the attack detection accuracy almost remains the same or even becomes better based on both BN and C4.5 classifiers. Constructing fewer features can also improve the efficiency of network intrusion detection.