PTC: Computation-Efficient Pyramid-Transformer-CNN with Global-to-Local Alignment for Medical Image Segmentation

Hybrid architectures combing Convolutional Neural Networks (CNN) encoder and Transformers encoder suffer two critical limitations: 1) unidirectional local-to-global feature injection, which constrains the contextual modeling capacity of Transformer encoder due to over-dependence on the inductive biases of CNN encoder, 2) the inefficient fusion designs that fail to bridge the semantic gap between two encoders while incurring high computational overhead. To overcome these limitations, we propose PTC (Pyramid-Transformer-CNN), a novel hybrid dual encoder architecture featuring three key innovations. First, a reverse feature injection mechanism (Global-to-Local) that guides the local feature extraction of CNN using Transformerderived semantic priors, enhancing global representation capacity. Second, pyramid input coupled with isolated hierarchical Transformer, balancing the abstraction levels between encoders. Third, a weight-free pyramidal fusion scheme eliminates complex fusion modules while maintaining feature consistency. We evaluated the proposed architecture on six public medical image datasets with four modalities. Compared with current methods, the experimental results show that the PTC achieves a new state-of-the-art performance. The extension of the architecture is further validated through hierarchical designed convlutional stems for 2D segmentation and a cross-scale stripe patch attention for 3D volumetric analysis. These adaptations maintain the computational advantages of PTC while consistently improving segmentation performance across modalities.