Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
October 13, 2025Open Access

Benchmarking Large Language Models for Multi-Language Software Vulnerability Detection

View Full Paper
Ask AI
Bookmark
Share

Authors

TZTing ZhangJimei UniversityCYChengran YangSingapore Management UniversityYSYindu SuZhejiang University of Science and Technology

Discussion

Loading...

Member takes

Implication

Empirical analysis reveals large language models struggle with software vulnerability detection, suggesting room for improvement via diverse techniques.

Key Points

  • Large language models face significant challenges in software vulnerability detection due to the complexity of the task.
  • The study utilizes a dataset of over 28,000 vulnerable functions across JavaScript, Python, and Java to benchmark model performance.
  • Various strategies like prompt engineering and instruction tuning are explored to enhance the effectiveness of large language models.
  • Findings indicate that improvements can be achieved through balanced data retraining and ensemble learning methods.

Cite This Study

Zhang et al. (2025) studied this question.

synapsesocial.com/papers/68ece2abd1bb2827d1297277https://doi.org/10.48550/arxiv.2503.01449
View Full Paper
Ask AI
Bookmark
Share

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study2024 · 1 citations
  2. 2A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Models2026 · 7 citations
  3. 3Using Large Language Models to Better Detect and Handle Software Vulnerabilities and Cyber Security Threats2024 · 29 citations
  4. 4To Err is Machine: Vulnerability Detection Challenges LLM Reasoning2024 · 18 citations
  5. 5Outside the Comfort Zone: Analysing LLM Capabilities in Software Vulnerability Detection2024