What type of study is this?

This is a Quantitative Study study.

October 10, 2025Open Access

Real-VulLLM: An LLM Based Assessment Framework in the Wild

Key Points

The framework demonstrates that LLMs can effectively detect vulnerabilities in real-world scenarios, enhancing software security.
An innovative scoring measure evaluates both accuracy and reasoning quality of vulnerability detection methods using LLMs.
By leveraging prompt designs tailored for vulnerability detection, the study shows potential for scalable implementation in software engineering.
A real-world vector data store derived from the National Vulnerability Database provides timely context to improve vulnerability assessment.

Abstract

Artificial Intelligence (AI) and more specifically Large Language Models (LLMs) have demonstrated exceptional progress in multiple areas including software engineering, however, their capability for vulnerability detection in the wild scenario and its corresponding reasoning remains underexplored. Prompting pre-trained LLMs in an effective way offers a computationally effective and scalable solution. Our contributions are (i)varied prompt designs for vulnerability detection and its corresponding reasoning in the wild. (ii)a real-world vector data store constructed from the National Vulnerability Database, that will provide real time context to vulnerability detection framework, and (iii)a scoring measure for combined measurement of accuracy and reasoning quality. Our contribution aims to examine whether LLMs are ready for wild deployment, thus enabling the reliable use of LLMs stronger for the development of secure software's.

Read Full Paperexternally

Bookmark

View Full Paper

Cite This Study

Safdar et al. (Sun,) studied this question.

synapsesocial.com/papers/68e8ed7aa1d181ff1b948145 https://doi.org/https://doi.org/10.48550/arxiv.2510.04056

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

Bookmark

View Full Paper