AI-Agent-Based Web Application Security Testing vs. Conventional Automated Security Testing: An Exploratory Case Study

Recent advances in Large Language Models (LLMs) and agent-based architectures have introduced new possibilities for automating web application security testing. Unlike traditional vulnerability scanners that rely mainly on signature-based detection, LLM-powered agents can analyze application context and dynamically adapt testing strategies. This exploratory case study examined two specific laboratory-defined workflows for identifying web application security weaknesses: a human-supervised agent-based workflow and a conventional vulnerability scanner workflow. The evaluation was performed in a controlled laboratory environment against the same target application and predefined ground truth, while each workflow retained its respective application-access, discovery, and authentication mechanisms. The agent-based framework was developed using Claude Code, the Model Context Protocol, and Burp Suite Professional, while the scanner-based approach utilized Qualys Web Application Scanning. Results were mapped to a predefined ground truth covering SQL Injection, Cross-Site Scripting, and Broken Access Control and reported using Recall, Precision, and false-positive counts where estimable. The findings describe what each completed workflow showed under the tested conditions, including coverage, tester effort, and validation workload, and outline directions for further evaluation.