Confusion-Aware Explainable Alert Triage for Network Intrusion Detection

Machine learning-based network intrusion detection, the explanation of detector decisions, and alert triage have each developed as a largely separate line of research. As a result, a detector’s aggregate performance score still tells a security analyst little about how far to trust an individual alert: it does not reveal how reliable the detector is for each attack type, whether the explanation of a decision is faithful, or on what calibrated basis some alerts should be automated and others escalated. We propose a confusion-aware triage framework, a calibrated and auditable per-attack-type triage layer that unifies these three concerns, which detection, explanation, and alert-triage research have pursued separately. From a trained detector’s predictions, the framework estimates the reliability of each attack type, validates the explanations of its decisions, and routes each alert, by that reliability, into automatic handling, analyst assistance, or escalation, presenting each escalated alert as a compact decision card. We evaluate the framework on UNSW-NB15 and CIC-IDS2017 with six detector families over five seeds. Per-attack-type reliability is estimable and stable across training runs, and transfers from validation to the held-out test fold, whereas a detector’s confidence overstates its reliability on the hardest attack types; the explanations are faithful for seven of the eight routed attack types; and routing by reliability escalates 27.9% of alerts while catching 75.7% of the detector’s errors, and abstains where the detector is already reliable. Routing by reliability matches routing by confidence on error coverage while remaining calibrated, per-attack-type, and auditable, and combining reliability with per-alert confidence catches more of the detector’s errors at the same workload. The framework is proposed as an analyst-facing decision aid, and its evaluation with security analysts is left to future work.