Integral quality assessment of machine learning models for cyberattack detection in network traffic

Machine learning models are widely used to detect cyberattacks through the analysis of network traffic. Traditional performance metrics do not account for the temporal characteristics of the data and may mask performance degradation within individual time intervals or under varying traffic intensity. Approaches based on key performance indicators, weighted geometric aggregation, and Bayesian methods allow individual performance characteristics to be combined; however, by themselves, they do not account for changes in model performance across predefined operating states. The developed integrated assessment methodology takes into account the frequency and certainty of erroneous decisions, as well as the degradation of model quality in specific operating states. A partial model quality indicator is calculated for each combination of time interval, target recall level, and traffic intensity mode. The indicator is formed by multiplicatively combining the classification score, the average calibrated confidence in correct decisions, the loss from erroneous decisions, and a time multiplier. The integrated indicator is obtained by averaging the partial indicators proportionally to the number of observations and subsequently adjusting for deviations below the weighted average. Experimental samples were generated using a single chronological data partition, eliminating the use of test records during model training, probabilistic calibration, and selection of classification thresholds. The methodology was evaluated on GothamDataset2025 using a test set of 1,777,237 network traffic records. Logistic regression, random forest, and histogram-based gradient boosting were compared across 18 operating states. At thresholds selected to provide validation recall of at least 0.50, random forest achieved the highest balanced accuracy of 0.693933 and the Matthews correlation coefficient of 0.466685. Histogram-based gradient boosting obtained the highest integral quality score of 0.264163. The methodology identified states with reduced or zero state-specific indicators that were not revealed by balanced accuracy and the Matthews correlation coefficient calculated over the entire test set, and changed the final model ranking. Operating states and aggregation parameters must be defined before assessment. The methodology supports the selection of cyberattack detection models under varying network traffic conditions.