Codeguard: Utilizing Advanced Pattern Recognition in Language Models for Software Vulnerability Analysis

Abstract Enhancing software quality and security hinges on the effective identification of vulnerabilities in source code. This paper presents a novel approach that combines pattern recognition training with cloze-style examination techniques in a semi-supervised learning framework. Our methodology involves training a language model using the SARD and Devign datasets, which contain numerous examples of vulnerable code. During training, specific code sections are deliberately obscured, challenging the model to predict the hidden tokens. Through rigorous empirical testing, we demonstrate the effectiveness of our approach in accurately identifying code vulnerabilities. Our results highlight the significant advantages of employing pattern recognition training alongside cloze-style questioning, leading to improved accuracy in detecting vulnerabilities in source code.