Fast concept-based counterfactual explanations for image classification

We propose Fast Concept-based Counterfactual Explanations (FCCE) for generating counterfactual explanations for CNN image classification models in real time. While counterfactual explanations are intuitive for humans, existing vision methods often face a trade-off between semantic interpretability and computational efficiency, limiting their widespread adoption in interactive decision support systems. FCCE addresses this challenge by operating in a semantic concept space and deriving a closed-form analytical counterfactual solution, with the remaining computational overhead dominated by standard feature extraction. To evaluate practical utility, we conducted a between-subject human study with 197 participants on a fine-grained model-failure identification task on the CUB dataset. Participants achieved 89.88% decision accuracy with FCCE, outperforming CVE (50.89%), ACE (68.47%), and ICE (82.95%), and reported higher perceived understanding, confidence, and satisfaction. Complementing the behavioral results, we assess fidelity and computational efficiency on CUB, MNIST, and ImageNet. Crucially, we demonstrate that FCCE counterfactuals remain highly faithful when mapped back to the original non-linear CNN, achieving a Relative Logit Gap below 0.3%, thereby providing strong empirical evidence for the fidelity of our counterfactual explanations. Finally, FCCE computes the analytical counterfactual solution in under 10 − 5 seconds on CPU, with an end-to-end local explanation overhead of approximately 10 − 2 seconds per instance. This is orders of magnitude faster than both optimization-based baselines and the representative source-conditioned diffusion-based generative counterfactual pipeline used as a latency reference. Overall, FCCE reconciles semantic interpretability with real-time performance and improves human decision-making about model reliability.