Attention-Augmented Deep Learning Architectures for Malware Detection Through System Call Sequence Analysis

The rise in sophisticated malware has exposed the limitations of traditional signature-based and static analysis methods, which often fail against obfuscated or evolving threats. Deep learning has shown promise in handling sequential data, such as API call sequences; yet, conventional models often struggle to capture critical subsequences within complex, variable-length inputs. This paper addresses this gap by systematically evaluating attention-augmented deep learning architectures for API sequence-based malware detection. We compare the performance of standard Convolutional Neural Network, Long Short-Term Memory (LSTM), and Gated Recurrent Unit models against their attention-augmented counterparts, while a Transformer serves as a robust baseline. Our results demonstrate that the attention mechanism consistently improves model robustness and interpretability. Crucially, attention-augmented models significantly reduced the False-Negative Rate (FNR), a critical metric in cybersecurity where missed detections are costly. The Attention-LSTM (A-LSTM) achieved the highest performance on Dataset 1, with an F1-score of 0.997 and an FNR of 0.001. Evaluated on the independent Mal-API-2019 test set, the A-LSTM demonstrated practical efficiency with an inference time of approximately 1.55 ms per sample and a memory footprint of 160 MB. Visual analysis of attention weights confirmed that the models focused on the most discriminative API calls. Overall, this work establishes a systematic evaluation framework that demonstrates the attention mechanism as a key component for improving the efficiency, robustness, explainability, and deployability of deep learning models in cybersecurity.