A Hybrid CNN-SelfAttention Model for Anomaly Detection in Encrypted Traffic Metadata

This study introduces a deep learning framework integrating convolutional neural networks and self-attention mechanisms for anomaly detection in encrypted network traffic, addressing the limitations of traditional methods reliant on payload inspection. Utilizing the UNSW-NB15 dataset, the approach preprocesses metadata to handle class imbalance, missing values, and categorical features, followed by training a hybrid model combining Conv1D layers for local feature extraction and self-attention for capturing long-range dependencies. The model achieves a test accuracy and F1-score of 0.9927, surpassing classical machine learning models (best F1-score of 0.9284) and a baseline LSTM-self-attention model (best F1-score of 0.7355). Its computational efficiency supports real-time applications, offering a privacy-preserving solution for detecting threats like DDoS attacks and malware. This research advances network security by providing a scalable and robust framework for encrypted traffic analysis.