H2-MAS: A Hybrid Heuristic-Multi-Agent Architecture for Semantic Bot Detection

Traditional Web Application Firewalls (WAFs) rely on static thresholds to detect automated threats. While effective against simple scripts, these deterministic rules struggle with “Ambiguous Traffic”—sophisticated bots that mimic human behavior and “Efficient Humans” (Power Users) who exhibit bot-like speed. In this paper, we introduce H2-MAS, a hierarchical security framework that combines a high-speed Random Forest classifier (Tier 1) with a Multi-Agent Cognitive Council (Tier 2) powered by Large Language Models (LLMs). Unlike standard “Black Box” LLM deployments, H2-MAS utilizes an adversarial “Prosecutor vs. Defender” protocol to resolve semantic paradoxes in real-time. We evaluated the system on a stratified dataset of 1000 high-uncertainty sessions (0.2≤p≤0.8). Through simulated evaluation on a stratified semantic test set, the results demonstrate that the proposed architecture has the theoretical capacity to achieve up to 97.6% accuracy on specific edge cases where traditional heuristics fail. Notably, the Cognitive Council demonstrated the theoretical capacity to reduce the False Positive Rate to 0.00% within this constrained evaluated set, validating that the “Defender” agent can successfully protect legitimate power users from erroneous blocking. This architecture offers a cost-effective paradigm for Semantic Security, prioritizing user experience without compromising threat detection.