Evidence-Grounded Autonomous Web Application Testing: A Scope-Aware Framework for Reliable AI-Generated QA Reports

Autonomous artificial intelligence agents powered by large language models (LLMs) are increasingly being explored for web application testing, usability evaluation, and software quality assurance. However, significant challenges remain in ensuring reliable defect detection, comprehensive test coverage, reproducible findings, and evidence-based reporting. This working paper introduces Evidence-Grounded Autonomous Quality Assurance (EG-AQA), a proposed framework for improving the transparency, reliability, and reproducibility of AI-generated web application testing reports. The framework combines task-oriented browser exploration, structured evidence collection, independent verification mechanisms, explicit testing-scope accounting, and uncertainty-aware reporting. The research examines how autonomous testing systems can distinguish verified software defects from unsupported interpretations, incomplete observations, environmental restrictions, and issues requiring human judgment. It also proposes quantitative evaluation methods for measuring defect-detection precision and recall, navigation coverage, task completion, finding reproducibility, and computational efficiency. The study uses an anonymized, limited observation from an early-stage Cohorvix testing report to illustrate challenges associated with incomplete exploration and the interpretation of automated testing evidence. A reproducible experimental methodology is proposed for future validation using controlled web applications, independently documented defects, repeated testing sessions, comparative baselines, and human adjudication. The paper contributes a structured research framework and evaluation protocol intended to support more accountable and evidence-grounded AI-powered website testing. The proposed improvements have not yet been validated through a completed comparative experiment. Publication status: Research working paper, Version 1.0. Research context: Cohorvix — https://cohorvix.com