Objective To develop and compare four explainable artificial intelligence methods to visualise the influence of electronic health records (EHRs) on predicting hip replacement risk. Methods and analysis We used a pretrained temporal graph-based convolutional neural networks (TGCNN) model to generate explainable graph visualisations through four methods: the original gradient-weighted class activation mapping (Grad-CAM) applied to graphs, a modified Grad-CAM using absolute weights (Grad-CAM (abs)), sliding element-wise multiplication of feature maps with patient graph inputs (fm-act) and of 3D convolutional neural networks filters/kernels with patient graph inputs (edge-act). These methods visually explain the TGCNN model’s predictions regarding a person’s risk of needing a hip replacement within 5 years, based on clinical codes from EHRs. We evaluated these models through human qualitative analysis studies, sensitivity quantification, edge detection bias and sparsity. Results The edge-act methods performed best in terms of graph sparsity and model sensitivity. Subgraph analysis indicated that prescriptions highly influenced predictions. Clinicians found the visualisations useful for explaining model predictions but too complex for clinical decision-making, particularly with extensive patient EHRs. Conclusions The fm-act and Grad-CAM (abs) methods led to graphs with zero sparsity; these graphs could be difficult to interpret if the patient has a long EHR history. The edge-act median method had the highest sparsity; therefore, this method might be the easiest to interpret for long EHR histories. We improved the explainability of hip replacement risk predictions using four post hoc methods on the TGCNN model. Further refinement could enhance their utility in clinical decision-making.
Hancox et al. (Thu,) studied this question.