Improving HRNet with a Focus on Practical Efficiency

The paradigm in computer vision has shifted from Convolutional Neural Networks (CNNs) to Transformers,yet recent work has revitalized CNNs by showing that modernized architectures can achieve state-of-the-art results.This study embraces this trend by improving the High-Resolution Network (HRNet), a prominent architecture fordense prediction tasks, focusing on practical performance. Previous HRNet enhancements often relied on the invertedbottleneck structure to reduce theoretical model parameters and FLOPs. However, this design is frequentlyinefficient on parallel hardware like GPUs. Its core component, depth-wise convolution, has low arithmetic intensity,which can slow down execution and increase memory consumption, creating a gap between theoretical and practicalefficiency. Our work aims to improve the performance of HRNet while maintaining its practical efficiency. We deliberatelyavoid the inverted bottleneck and instead integrate four established, computationally inexpensive techniquesinto HRNet. Our proposed modifications include: (1) the multi-path down-sampling from ResNet-D to preserve information;(2) the optimized block component layout from ResNetV2; (3) the Cross Stage Partial Network (CSPNet)for richer gradient flow; and (4) selective integration of attention mechanisms to enhance representational power efficiently.Our models were evaluated on image classification and human pose estimation. The results demonstrated thatthe optimal enhancements are task-specific: a combination of all four techniques was best for classification. Furthermore,practical efficiency evaluations on an NVIDIA A100 GPU revealed that while the inverted bottleneck structurecaused a severe drop in inference throughput and out-of-memory (OOM) errors during training, our proposed modelsuccessfully maintained high execution efficiency. Meanwhile, only the down-sampling improvement excelled at humanpose estimation. This research provides a concrete method for improving HRNet’s baseline performance withoutsacrificing its practical efficiency. It also offers valuable insights for future CNN design by presenting a successfulimprovement strategy that is not dependent on the inverted bottleneck structure.