Insights into anomaly-based intrusion detection systems usability. A case study using real http requests

Intrusion detection systems based on anomalies (A-IDS) are crucial for detecting cyberattacks, especially zero-day attacks. Numerous A-IDS proposals in the literature report excellent performance according to established metrics and settings in a laboratory. However, finding systems implementing these proposals in real-world scenarios is challenging. This work explores, through a case study, the suitability of performance metrics commonly used in the scientific literature to real-world scenarios. Our case study will consider a Web attack detector based on URIs and a real, large-scale dataset. Our results show significant limitations in the performance metrics commonly used to select the system's operating point and its practical use in real-world scenarios.