Explainable AI for raising confidence in deep learning‐based tumor tracking models

Background Recently, tumor position monitoring using fluoroscopic images acquired during volumetric modulated arc therapy (VMAT) delivery has become available in a research setting. Accurate tracking during stereotactic body radiotherapy (SBRT), using VMAT, for lung tumors can help to ensure that the tumor is only irradiated when it is inside the planning target volume. Purpose Traditionally, template matching is used to determine the tumor position, but with low tracking rates. A deep learning‐based approach has the potential to improve this, but as deep learning is considered a “black box,” it would be desirable to know when to trust the predictions made by the model. Methods We investigate the reliability and effectiveness of four explainable AI (XAI) methods (Guided Backpropagation (GBP), Layer‐wise Relevance Propagation (LRP), DeepLIFT and PatternAttribtuion) to highlight the most relevant features for a deep learning‐based 2D markerless lung tumor tracking model. The experiments are conducted on two phantoms and six clinical patients with small lung tumors (0.23–2.93 ). Both quantitative and qualitative evaluation is conducted to assess the suitability of the selected XAI methods for tumor tracking. Results Our findings suggest that out of the four selected XAI methods, only GBP and DeepLIFT demonstrate a reliable and consistent behavior across all patients and phantoms; LRP shows good performance in the phantom setting but has lower qualitative results on the clinical data. Conclusions Based on our results, we argue that GBP and DeepLIFT can be used out‐of‐the‐box to explain deep learning‐based tracking models for SBRT using VMAT. Further investigation is needed to develop a robust measure of the model's reliability in clinical practice during treatment delivery.