Hierarchical confusion matrix for classification performance evaluation

Abstract This study proposes the novel concept of hierarchical confusion matrix, opening the door for popular confusion-matrix-based (flat) evaluation measures from binary classification problems, while considering the peculiarities of hierarchical classification problems. The concept is developed to a generalised form and proven its applicability to all types of hierarchical classification problems including directed acyclic graphs, multi-path labelling, and non-mandatory leaf-node prediction. Finally, measures based on the novel confusion matrix are used for three real-world hierarchical classification applications and compared to established evaluation measures. The results, the conformity with important attributes of hierarchical classification schemes and its broad applicability justify its recommendation.