Entropy-Based Metrics for Evaluating Supervised Machine Learning Models: A Scoping Review

In supervised machine learning, selecting appropriate performance metrics is crucial, as they directly influence model comparison, interpretation, and selection. Although traditional metrics such as accuracy, precision, recall, and F1 score are widely used, especially in complex or imbalanced classification tasks, they may not fully reflect prediction uncertainty or class-level variation. Thus, entropy-based metrics have emerged as complementary tools for evaluating uncertainty and information content in model outputs. This scoping review analyses 40 peer-reviewed empirical studies published between 2016 and 2024 that implement entropy-based metrics in supervised classification. The review determines the main entropy-based metrics used in the literature, examines how they are combined with or compared to traditional metrics, and emphasizes the contexts in which they provide added value. Entropy-based and information-theoretic measures can complement traditional classification metrics by providing additional information about uncertainty, calibration, distributional mismatch and class-level prediction behaviour, particularly in imbalanced or high-risk classification settings. Three barriers to wider adoption recur across the reviewed studies: computational cost, parameter sensitivity, and the absence of benchmarking standards.