Answers are grounded in this paper and cite their sources.
Large Language Models (LLMs) are now built into many everyday applications, which exposes them to jailbreak and prompt injection attacks that try to override safety rules or hijack the instructions a model follows. This paper presents a machine learning framework that screens prompts and flags likely malicious ones before they reach the model. We compare Logistic Regression, a Linear Support Vector Machine (SVM) and a fine-tuned DistilBERT classifier on a combined dataset of 24,530 prompts drawn from GuardrailsAI and Neuralchemy, prepared through label normalization, duplicate checking and stratified splitting. After the decision threshold was tuned on validation data, DistilBERT reached 95.56% accuracy, 93.74% precision, 88.77% recall and a 91.18% F1-score on the held-out test set. On a separate collection of 300 JailbreakBench-associated prompts, however, accuracy fell to about 82% and 55 of 200 attack prompts went undetected, which shows how much harder diverse adversarial inputs are than in-distribution data. The results suggest that transformer-based screening can serve as a useful additional security layer for LLM applications, provided it is validated on external benchmarks and combined with other defences.
No takes yet. Share an insight, caveat, or question.
Sonawane et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: