Comparative study demonstrates structured evidence grounding improves investigation clarity and recall in security operations centers, indicating a trade-off between thoroughness and runtime.
Security Operations Centers (SOCs) must investigate alerts by retrieving heterogeneous evidence, correlating events, testing hypotheses, and producing conclusions that analysts can trust and evaluate. Large language models (LLMs) have recently been applied to alert triage, log analysis, and autonomous cyber investigation, but existing systems and evaluations often emphasize individual capabilities rather than the integrated investigation workflow. This work presents Argus, an AI-assisted SOC investigation prototype that structures LLM-driven investigation around bounded planning, Zeek based evidence retrieval, an evidence ledger, evidence-grounded report generation, deterministic citation verification, and report revision. We evaluate whether this additional structure provides measurable benefits beyond simply exposing an LLM to security telemetry. The evaluation uses a paired four condition design on eleven modular investigation cases derived from the OTRF APT29 Detection Hackathon dataset: C1 deterministic investigation, C2 LLM only investigation, C3 LLM with Zeek access, and C4 Argus. All LLM conditions use the same Qwen3-4B-GGUF model and fixed generation settings. Primary outcomes are investigation time, explanation clarity, and grounding precision, attack-step recall and hallucination rate are analyzed as supporting outcomes. Across the 11 cases, Argus obtains the highest mean clarity score (3.57/5) and attack-step recall (0.677), and significantly outperforms C3 in clarity, grounding precision, and attack-step recall after Holm correction. Argus does not demonstrate a runtime advantage over C3 and requires substantially more time on the median case. Hallucination rate does not differ significantly across conditions. These results support a narrower conclusion: structuring LLM investigation can improve the completeness and analyst-oriented quality of investigations, but these gains introduce computational and temporal cost. Argus therefore represents an empirical study of the trade-off between investigation efficiency, evidence-grounded credibility, and explanation clarity rather than a claim of universal superiority over conventional investigation.
No takes yet. Share an insight, caveat, or question.
El Batoul Djaafar (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: