AgentForensics: Exploring the Real-Time Prompt Injection Detection and Forensics Threats in LLM Agents

Abstract: With LLM agents increasingly deployed to autonomously processed external content like web pages, emails,documents, and API responses become targets of indirect prompt injection attacks where malicious instructions embeddedin external content hijack the agent’s behavior. Existing defenses focus primarily on filtering user-layer inputs, leavingthe wider attack surface unaddressed. In this paper, we present AgentForensics, an open-source security frameworkthat monitors entire LLM agent sessions in real time and detects prompt injection attacks wherever they originate. Oursystem implements a five-stage detection pipeline combining seven heuristic regex rules and one Unicode script-ratiocheck, a pre-trained DistilBERT-based ML classifier, instruction boundary pattern matching, semantic drift analysis, andsliding-window multi-turn detection. We evaluate AgentForensics across two independent benchmarks ARPI bench (7,560payloads) and deepset/prompt-injections and achieve 100% detection on 7,763 injection payloads with zero false positivesacross 343 benign samples. F1 score of 1.0 is reported for the deepset benchmark, which contains both injection andbenign classes. Our framework requires minimal-integration support to existing agent logic and integrates natively withOpenAI, Anthropic, LangChain, AutoGen, and Claude Desktop via MCP.