AgentShield Bench v2: Evaluating Memory Security, Persistent Jailbreaks, and Cross Session Compromise in Autonomous LLM Agents

AgentShield Bench v2 is a benchmark framework for evaluating persistent memory security vulnerabilities in autonomous Large Language Model (LLM) agents. Unlike traditional AI security benchmarks that primarily focus on prompt injection and single-session adversarial attacks, AgentShield Bench v2 investigates how malicious information can be stored in long-term memory, persist across conversational boundaries, and influence future agent behavior. The benchmark introduces a dataset of 320 memory-security attack scenarios spanning eight threat categories: False Memory Injection, Memory Corruption, Persistent Jailbreaks, Goal Manipulation, Privilege Escalation, Cross-Session Persistence, Trust Exploitation, and Recovery Evaluation. AgentShield Bench v2 proposes a memory-focused evaluation methodology based on five metrics: Memory Integrity Score (MIS) Persistence Score (PS) Behavioral Compromise Rate (BCR) Recovery Effectiveness Score (RES) Memory Security Score (MSS) Experimental evaluation was conducted using GPT-4o-mini and Llama-3.1-8B-Instant, providing comparative analysis of memory poisoning resilience, defense effectiveness, and cross-session compromise behavior. A key finding of this work is the identification of the Key Mismatch Gap, an evaluation phenomenon in which behavioral compromise can occur despite memory-integrity metrics reporting near-perfect security. This highlights the limitations of exact-match memory evaluation techniques and motivates future research into semantic memory-security assessment. This release contains the research paper associated with AgentShield Bench v2 and serves as a citable technical report from Evalyze Labs.