From Risk Scores to Budget-Constrained Alert Streams: Temporal Evaluation of Flow-Level Network Intrusion Detection

An intrusion detector may rank attacks well yet exceed its false-positive allowance when a historical threshold is applied to later traffic. We audit this score-to-alert step in five-run forward-time replays of CICIDS2017, UNSW-NB15, and NF-ToN-IoT-v3, reporting threshold provenance, false positives per million benign flows (FP@1M =FPR×106), budget overrun, and local attack coverage. Historical benign and past-only rolling thresholds are contrasted with retrospective label-assisted diagnostics. CICIDS2017’s benign-only first fold is a cold-start failure; a post hoc subset of later, attack-informed folds attains 0.397±0.071 recall with 13,749±5262 realized FP@1M at a nominal 10 k budget. On NF-ToN-IoT-v3, historical calibration yields 0.403±0.007 recall and 18,477±1664 realized FP@1M; at an assumed attack prevalence of 10−4, this corresponds to about 2.18 true attacks per 1000 alerts. The UNSW-NB15 diagnosis changes under an alternative capture-session split. Random–forward gaps remain confounded by training volume and composition. These findings support joint reporting of threshold provenance, realized alert load, uncertainty, and local coverage; they do not establish a new detector or production utility.