Refining CVE-to-CWE mapping with enhanced attention in BERT-based models

In this study, we introduce CySecBERT-ARD, an advanced approach for classifying software vulnerabilities that maps Common Vulnerabilities and Exposures (CVE) to Common Weakness Enumerations (CWE). Our approach is to use a pretrained transformer-based model CySecBERT tailored for cybersecurity contexts, the model is enhanced with additive attention and relative position encoding which allow for a deeper understanding of the vulnerability descriptions of CVE by capturing the contextual relationships. Our approach achieves an impressive accuracy of 91.34% and F1-score of 91.32% during the evaluation and testing phase compared to the base models. The results demonstrate the potential of CySecBERT-ARD in enhancing the efficiency and effectiveness of vulnerability classification.