Answers are grounded in this paper and cite their sources.
Recently, Vision Transformers show superiority and promise on various vision tasks owing to the unique long-range modeling capability of the self-attention mechanism. Although effective, it suffers from redundant attention on non-important regions like background, which inevitably increase the computational overhead. In this paper, we design a deformable poll and pool (DPnP) module to compress the key, value feature maps to retain the important features at fine granularity and compress the rest features at coarse granularity. Applying the DPnP module to the shunted self-attention significantly reduces attention redundancy, allowing for more computational space allocated to key foreground object regions, achieving more efficient shunted self-attention. The experiments demonstrate its effectiveness on the Intel-Image-Classification dataset, obtaining better top-1 accuracy, achieving 0.23% higher with 10.6% reduction of parameters.
No takes yet. Share an insight, caveat, or question.
Wu et al. (2024) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: