Accuracy Rejection Normalized-Cost Curves (ARNCCs): A Novel 3-Dimensional Framework for Robust Classification

Machine learning (ML) offers several supervised learning algorithms to build classifiers for developing accurate decision support systems. However, the selection of robust classifier for reliable decision making in healthcare domain is three-fold: accuracy, refraining for low confidence decisions, and the cost of decisions. In the field of medical science, there are costs associated with the incorrect and the refraining from decisions, which can have negative implications in devising adequate therapeutic interventions. For example, it may be life threating if a cancer patient is declared as healthy one (misclassification cost) or decision remains pending for some time (rejection cost). In this work we proposed the concept of Accuracy Rejection Normalized-Cost Curves (ARNCCs), which is an extension of Accuracy Rejection Curves (ARCs); a three-dimensional visualization technique to demonstrate the strengths and weaknesses of classification algorithms over different rejection regions and normalized-cost (NC) to select the robust classifier. ARNCCs method holds ARCs plot on two dimensions, in addition it computes NC at third dimension against ratio of false positive costs and ratio of rejection costs obtained at different rejection rates. The proposed three-dimensional graphs have the potential to answer a variety of questions regarding accuracy, rejection rate and NC of a classifier. Six publicly available cancer datasets (four breast cancer and two lung cancer) having clinical parameters obtained from ML repository of University of California, Irvine (UCI) were used to assess the performance of proposed ARNCCs in this study. Empirical results show that ARNCCs provide broad range of decisions to choose the desired parameters (accuracy, rejection rate and NC) for further necessary actions as compared to traditional ARCs method. ARNCCs framework has the ability to more logically compare the performances of classification algorithms in terms of accuracy, rejection rate and NC based scenarios.