GaPaTTA: Gaussian Entropy-Guided Prompt Placement for Test-Time Adaptation in Semantic Segmentation

Abstract Test-time adaptation (TTA) aims to improve model robustness under domain shifts without access to source data–an essential capability for real-world applications such as autonomous driving and robotics. Existing TTA methods for semantic segmentation often rely on stochastic techniques like Monte Carlo dropout or augmentation-averaged predictions to estimate uncertainty or stabilize outputs. However, these approaches typically require multiple forward passes, which are computationally expensive and limit real-time applicability. We propose GaPaTTA, a lightweight and deterministic TTA framework built on SegFormer. Unlike previous methods, GaPaTTA adopts a single forward pass with a traditional augmentation strategy, avoiding repeated inference required by ensemble-based TTA approaches. Key innovations include: (1) Grad-CAM-based global prompt placement identifies the most relevant encoder layers for adaptation; (2) Gaussian entropy-guided local prompt injection selects the top-K most uncertain pixels; (3) Shannon entropy-based filtering suppresses unreliable pseudo-labels; and (4) cross-stage consistency aligns mid- and high-level features for structural coherence. Experiments on ACDC (A-Fog, A-Night, A-Rain, A-Snow), Cityscapes-Foggy (CS-Fog) and Cityscapes-Rainy (CS-Rain) demonstrate that GaPaTTA consistently outperforms previous TTA methods in mean intersection over union (mIoU) while reducing inference time by over 50%. The source code is available at https://github.com/ml4papers/GaPaTTA .