Counterfactual Causal-Effect Intervention for Interpretable Medical Visual Question Answering

Medical Visual Question Answering (VQA-Med) is a challenging task that involves answering clinical questions related to medical images.However, most current VQA-Med methods ignore the causal correlation between specific lesion or abnormality features and answers, while also failing to provide accurate explanations for their decisions.Moreover, VQA-Med methods suffer from the common language bias problem in generic VQA.To explore the interpretability and language bias of VQA-Med, this paper proposes a novel CCIS-MVQA model for VQA-Med based on a counterfactual causal-effect intervention strategy.This model consists of the modified ResNet for image feature extraction, a GloVe decoder for question feature extraction, a bilinear attention network for vision and language feature fusion, and an interpretability generator for producing the interpretability and prediction results.The proposed CCIS-MVQA introduces a layer-wise relevance propagation method to automatically generate counterfactual samples for improving interpretability and alleviating language bias.Additionally, CCIS-MVQA applies counterfactual causal reasoning throughout the training phase to enhance interpretability and generalization.Extensive experiments on three benchmark datasets show that the proposed CCIS-MVQA model outperforms the state-of-the-art methods.Enough visualization results are produced to analyze the interpretability and debasing performance of CCIS-MVQA.