A Codebert Based Empirical Framework for Evaluating Classification-Enabled Vulnerability Prediction Models

Software vulnerabilities are discovered quite frequently and cause substantial damage and security breaches. In the realm of cybersecurity, the detection of vulnerabilities plays a pivotal role in safeguarding critical systems and data. In this paper, we evaluate the efficacy of the source code embeddings obtained from the pre-trained CodeBERT model in detecting vulnerable code snippets and perform a comparative analysis with different data pre-processing and ML classification algorithms to recommend the best techniques for vulnerability detection. Precisely, we investigated the predictive ability of 540 different models developed using the CodeBERT embeddings, five feature selection techniques, a class balancing technique, and fifteen classification algorithms. Unlike regular deep learning-based models, these models are quick to implement and require minimal computational resources. The models are evaluated on robustness in detecting codes with known and previously unseen vulnerabilities. The developed models display high average accuracy and AUC, proving the efficiency of the CodeBERT embeddings in capturing vulnerabilities. We find that ensembles of decision trees (RF and EXTR) perform the best for vulnerability detection, which can be attributed to the advantages of ensembling. Furthermore, the effectiveness of class balancing and feature selection techniques as data pre-processing techniques is evaluated using box plots and the Friedman test. The significance of this research lies in its contribution to the field of cybersecurity by introducing reliable and efficient vulnerability detection methods.