Large language models (LLMs) are increasingly employed within Security Operations Centres (SOCs), including SOC for Digital Risk Protection (DRP), yet their outputs often exhibit partial coverage, hallucinations, verbosity, and lack of localized insights. This article proposes a hybrid reasoning pipeline that combines multimodal LLMs with stable human-curated references to mitigate these issues, and is distinct from standard retrieval-augmented generation because offline, human-curated references are applied as an explicit decision-time override rather than used solely as supportive retrieved context. We introduce a step-by-step process that incorporates multi-vantage crawling for evasive content, deterministic prompts to manage inconsistency, and a structured approach to override or refine the model’s classifications when local brand knowledge contradicts global assumptions, together with an analyst-governed escalation loop that records when and why overrides occur in external-SOC DRP settings. Empirical evaluations with multiple commercial and open-source model providers show that this method significantly boosts scam detection accuracy, lowers token costs through caching, and reduces misleading outputs by adopting curated domain data, including comparisons against a RAG-only configuration and classical non-LLM baselines. Results underline how offline reference injection fosters a reliable collaboration pattern that harmonizes automated tasks with human expertise, thereby enhancing scalability and trust in real-world SOC environments.
Xu et al. (Thu,) studied this question.