Answers are grounded in this paper and cite their sources.
Systematic evaluation reveals non-text attacks can bypass LLM safety defenses, suggesting vulnerabilities in current models.
Large language models (LLMs) are becoming better at defending against text-based prompt injection. However, we find that the defense can become much weaker when attack instructions are placed in non-text channels, such as image metadata, document structures, or hidden text elements. In this paper, we test five attack types: single-turn text injection, mutation-based attacks, multi-turn conversational attacks, multimodal hidden injection, and adaptive evolutionary attacks. We use the same testing framework for all of them. We run the experiments on three models: GLM-5.2, GLM-4.5, and GLM-4-Flash. The results show that attacks outside plain text can still bypass existing defenses. For GLM-5.2, the defense score drops from 100/100 on standard text injection to 26.7/100 on multimodal injection, giving a 73% bypass rate. Mutation attacks reach a 35.2% bypass rate, while multi-turn attacks reach 22%. We release our testing framework, agent-redteam, as an open-source tool. It contains 2,319 attack samples across 14 test suites covering the OWASP LLM Top 10.
No takes yet. Share an insight, caveat, or question.
Jiehan Zhu (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: