Dependability analysis and hardening of vision transformers against soft errors

Abstract The deployment of Vision Transformers (ViTs) in safety-critical domains needs a clear understanding of their resilience to soft errors, since their specific layer-level vulnerabilities are currently insufficiently characterized. This work presents a dependability analysis of the ViT-Base architecture against injection-induced soft errors. Using a high-fidelity, software-level fault injection methodology with custom CUDA kernels, the study injects random bit-flips directly into the IEEE 754 binary32 floating-point representation of the intermediate data tensors resulting from the Transformer modules to quantify model accuracy degradation across increasing bit error rates. As a primary result, a vulnerability map across ViT layers is presented, confirming that the results of normalization and fully connected layers exhibit critical sensitivity to soft errors. To address these vulnerabilities, the work evaluates targeted hardening strategies. These include Fault-Aware Training (FAT), applied both globally and selectively to linear layers, as well as practical runtime mitigations such as range-based value clipping and filtering of non-numeric values. The findings demonstrate that these software-only approaches can significantly protect model accuracy.