NeXtSwin-X: dual-branch cross-attention fusion of ConvNeXt and swin transformer for accurate brain tumor classification from MRI and CT

Accurate brain tumor classification from neuroimaging modalities such as Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) is critical for clinical diagnosis. However, fine-tuning large deep learning models on moderately sized medical datasets often demands extensive computational resources. This work demonstrates that strategic transfer learning can address this challenge. We propose a dual-branch architecture that combines two pre-trained backbones: a ConvNeXt to capture local patterns and a Swin Transformer to model global contextual dependencies. A cross-attention mechanism fuses these complementary features, while a differential learning rate strategy accelerates convergence. Extensive evaluation on eight publicly available brain MRI and CT datasets spanning binary and multi-class classification tasks shows that our model outperforms state-of-the-art baselines on most datasets and metrics and matches the best-performing models in remaining cases, all with substantially fewer training epochs. The main contribution of this work is the introduction of a dual-branch fusion architecture coupled with a differential learning rate strategy, enabling efficient adaptation of pre-trained backbones for medical image analysis.