Discriminative Index: A Novel Indicator for Evaluating Machine Learning Algorithms in Laboratory Medicine

Background: Machine learning models are increasingly applied in clinical practice. However, conventional performance metrics such as sensitivity, specificity, and accuracy do not capture the structural characteristics of predictive probability distributions, limiting their utility for optimizing decision thresholds and supporting clinical decision-making. Methods: We propose the D-index, a novel metric designed to quantify asymmetry in predicted probability distributions by measuring the difference in skewness between positive and negative predictions. The D-index was evaluated using simulated datasets with skewness-dominant and kurtosis-dominant distributions at three performance levels (AUROC = 0.7, 0.8, and 0.9). Its practical utility was further validated using real-world MALDI-TOF mass spectrometry data for predicting vancomycin resistance in Enterococcus faecium, comparing three machine learning algorithms: deep learning, XGBoost, and random forest. Results: In simulated data, the D-index increased with model performance only in skewness-dominant distributions, demonstrating selective sensitivity to distributional asymmetry. In real-world data, the D-index distinguished models with similar conventional performance but different distributional structures. Models with higher D-index values retained a larger proportion of confidently classified cases while achieving meaningful accuracy improvements under a gray-zone strategy. In contrast, models with lower D-index values required exclusion of more cases to achieve comparable or greater accuracy gains. Conclusions: The D-index provides complementary information beyond traditional performance metrics by capturing structural properties of predictive distributions. It enables identification of models that are better suited for gray-zone strategies and supports more reliable clinical decision-making. This distribution-aware approach offers a practical tool for improving the safety, efficiency, and clinical utility of machine learning models in biomedical applications.