Automatic Software Vulnerability Analysis through a Multi-Head Attention and Long Short-Term Memory Approach

As Software Vulnerabilities continue to grow, they become a major aspect of software quality reflected in the increasing number of discovered and predicted code bugs. Analysts play a crucial role in localizing vulnerabilities yet security analysis still requires a significant manual effort to fix and repair functions. In this paper, a proposed method combines Multi-Head Attention (MHA) and Long Short-Term Memory (LSTM). This combination allows the model to handle variable-length input sequences a crucial capability for applications where input lengths vary. Such models can effectively learn to recognize patterns and dependencies in sequential data. However, the MHA model becomes more complex when interpreting the contribution of each head to the final prediction, posing a challenge, especially when dealing with limited training data. The increased model complexity from using MHA can lead to overfitting. The experimental results presented in the paper demonstrate high-level performance on the SARD dataset, achieving an accuracy of 99.00%. This performance surpasses other existing models, such as Deep Neural Network, Recurrent Neural Network, and Graph Neural Network.