Joint Spatiotemporal Adversarial Attacks on Video Transformer Models Through XAI-guided Perturbation

The widespread deployment of video transformer models in action recognition systems necessitates a comprehensive understanding of their vulnerability to adversarial attacks. Unlike traditional CNN-based video models, transformers process spatiotemporal dependencies through self-attention mechanisms, creating a different vulnerability profile to adversarial attacks. This study presents an investigation of adversarial robustness in video transformers. We develop a novel joint spatiotemporal attack method that precisely targets the attention mechanisms of video transformers. By simultaneously perturbing both spatial and temporal features, our method achieves a 76.30% in ASR on the Kinetics-400 dataset, outperforming frame-wise attacks and state-of-the-art query-based attacks. To interpret the mechanisms underlying these attacks, we introduce quantitative metrics based on Explainable AI (XAI) analysis. Spatial analysis reveals systematic disruption of attention patterns, with adversarial examples showing median SSIM scores of 0.353. Temporal correlation analysis also demonstrates severe degradation in attention coherence across frame sequences. Through experiments comparing previous attack methods, including common corruptions benchmark, frame-wise attacks, sparse attacks, and recent V-BAD attacks, we demonstrate that our proposed method is more effective in transformer-based video models. This study further examines the adversarial training strategy against the selected attacks. To promote reproducibility and facilitate future research, we provide our methods and analysis tools through a public GitHub repository. These findings underscore the effectiveness of jointly considering spatial and temporal dimensions when developing adversarial attack strategies and defense mechanisms for video AI models.