Development of machine-learning based app for anomaly detection in CMSWEB

This paper presents an anomaly detection system for CMSWEB services, a critical component of CMS infrastructure supporting over two dozen web services. The system employs deep learning techniques to enhance service performance and reliability by detecting anomalies in real time. Multiple autoencoder-based models such as Hybrid CNN-LSTM, Hybrid CNN-GRU, GRU, LSTM, CNN, and a fully connected autoencoder were trained using data collected from the CMS Monit infrastructure. Hyperparameter tuning and continuous learning were implemented to adapt models to dynamic service behaviors. Anomaly detection thresholds were derived using statistical methods, including mean + 1.5 standard deviations, median absolute deviation, and the 95th and 99th percentiles of reconstruction errors. The Hybrid CNN-LSTM model demonstrated superior performance, achieving the lowest RMSE and MAE and the highest R-squared values. The developed app provides real-time insights, visualizations, and automated alerts for anomalies. This work significantly contributes to improving CMS operational efficiency by reducing downtime and enhancing system stability. Future research will focus on refining the monitoring dashboard, expanding anomaly insights, enabling user interaction for model training, and integrating expert feedback to further optimize detection processes.