VProtoNet: vision transformer-driven prototypical networks for enhanced interpretability in chest x-ray diagnostics

Deep learning-based methods have achieved significant improvement in accuracy in diagnosing lung diseases utilizing Chest X-Ray. However, their black-box nature and lack of interpretability reduce the confidence among physicians in the reliability of machine-generated decisions, which consistently limits their application in clinical practice. In this paper, we propose a novel interpretable deep learning model VProtoNet, which can produce heatmaps that display important diagnostic image features of lung diseases and reveal how the model makes decision based on them. VProtoNet generates heatmaps by comparing the features extracted by Vision Transformer with the prototypes, each of which signifies a typical part of a Chest X-ray image, learned within the model. Further, we simplify the heatmap into a single similarity score that can be used as the basis for model classification diagnosis. To verify the effectiveness of our model, we applied our method to Chest X-ray 14 dataset and achieved an accuracy of 72.35%. Also, we analyzed the feature maps generated by our model during the classification process, discovering that they indeed intuitively demonstrate the model's recognition and understanding of the diseased areas, which enables physicians to better comprehend the model's decision-making process.