Feature Fusion for Malware Traffic Detection Using Image Visualization and Transfer Learning

Malware threats have become increasingly sophisticated and more complex, making detecting and preventing them difficult. Traditional malware detection methods, such as signature-based detection, are no longer effective against new malware variants. Deep learning, especially convolution neural networks, has evolved significantly as end-to-end malware detection without much requirement of domain expert analysis for feature engineering. This research paper proposes a new image visualization-based malware detection and classification using transfer learning and machine learning classifiers. At first, the network traffic dumps (PCAPs) are transformed into grayscale images. After that, pre-trained CNN models, including VGG16, VGG19, EfficientNetB0, EfficientNetB1, and Xception, are customized to extract the textural features via transfer learning by adding a fully connected layer of different dimensions. Feature fusion is performed by stacking the extracted feature maps from different pre-trained models to form a single feature map. Finally, the fused feature maps are fed to train three machine learning models to obtain the best classifier. The hyperparameter optimization of all classifiers is performed using grid-search and randomized search algorithms. This study performs intensive experiments using 913 public image datasets, and the proposed method obtained 94.71% accuracy, which is an excellent performance even using a smaller dataset. The experiment results reveal that the proposed method is well-suited to deal with live malware traffic in real-world industrial IoT environments.