Answers are grounded in this paper and cite their sources.
Published results for process-level anomaly detection on the HIL-based Augmented ICS (HAI) dataset range from below 0.3 to above 0.95. This extended version of a conference paper, which reported an accuracy of 0.9996 for a Long Short-Term Memory (LSTM) detector on HAI 20.07, measures how much of this range is due to the evaluation protocol and metric rather than to the detectors. The same trained models are evaluated changing one choice at a time: input definition, data split, class imbalance treatment, threshold selection, point adjustment and evaluated segment. In the conference setting, a rule that copies the previous label scores as high as the LSTM, and a random split of the samples alone yields near-perfect scores. On HAI 23.05, a test-tuned threshold and point adjustment double the reported score of the same detector and compress the differences among six detectors, and random anomaly scores obtain a higher point-adjusted F1 than all of them. Under deployment-style protocols, the best semi-supervised detectors outperform supervised detectors trained on attacks against a subset of the control loops, but none meet a budget of one false alarm per day on the test data. A checklist of evaluation choices, code and data splits are released.
No takes yet. Share an insight, caveat, or question.
Giracca et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: