Parameter-Efficient Tuning for Fine-Grained Recognition via Channel-wise Importance Equalization and Diversity Navigation

Parameter-efficient tuning (PET) has achieved promising performance on various downstream vision tasks. Despite their effectiveness for general classification, existing PET approaches neglect the over-concentration of channel-wise saliency and the feature redundancy of pre-trained models during fine-tuning, thus leaving much room for improvement when applied to the downstream fine-grained recognition tasks. To address these issues, we propose a novel parameter-efficient tuning approach tailored for fine-grained recognition (FG-PET). Specifically, FG-PET first employs a Channel-wise Importance Equalization (CIE) module. It suppresses the concentrated salient channels while strengthening the remaining majority ones during fine-tuning, notably mitigating the over-concentrated saliency, thus evoking more channels within pre-trained models to deliver abundant local visual clues. Furthermore, FG-PET develops an Efficient Navigator for Diversity (EFIND) by introducing a center-based loss and orthogonal constraints on features generated from distinct attention heads. It alleviates the redundancy between different attention maps, thus enforcing the models to explore diverse subtle visual differences in various discriminative local regions, which are critical for fine-grained recognition. Extensive experimental results on five public fine-grained benchmarks based on distinct ViT models demonstrate that the proposed method remarkably boosts the performance of existing PET approaches, and generalizes well to general classification tasks. We will release the source code of our work upon acceptance.