Adapting Pretrained Transformer Blocks Within a Convolutional Framework for Slice-Based Brain MRI Segmentation and Deformable Registration

Accurate brain-tumour segmentation and longitudinal magnetic resonance imaging (MRI) registration require representations capable of capturing both local tissue characteristics and long-range anatomical dependencies. Inspired by the hierarchical local-to-global processing and multiscale information integration of biological visual systems, this study proposes a biomimetic Language-Transformer and Convolutional (LTC) framework, implemented as an attention-based U-shaped network (ALUN). The architecture integrates convolutional feature extraction with Transformer representations adapted from Bidirectional Encoder Representations from Transformers (BERT) and the Large Language and Vision Assistant (LLaVA). Its convolutional pathways emulate the extraction of local visual patterns, whereas attention-based processing and skip pathways facilitate global contextual integration and the preservation of fine anatomical information. BERT-ALUN and LLaVA-ALUN were evaluated in separate slice-based 2D segmentation and deformable-registration configurations using axial images extracted from BraTS and BraTS-Reg, respectively. The registration configuration estimates a dense two-component in-plane displacement field and does not estimate a full 3D transformation. Performance was assessed using the Dice similarity coefficient (DSC) and sensitivity, complemented by architectural ablation and image-noise experiments. For segmentation, LLaVA-ALUN achieved unweighted mean DSC and sensitivity values of 84.96% and 82.01%, respectively, compared with 80.59% and 80.03% for BERT-ALUN. For registration, LLaVA-ALUN obtained mean region-based DSC and sensitivity values of 78.52% and 75.24%, compared with 71.96% and 72.21% for BERT-ALUN. These results represent mean DSC improvements of 4.37 and 6.57 percentage points for segmentation and registration, respectively. LLaVA-ALUN also achieved higher DSC and sensitivity under the evaluated noisy-image conditions. The ablation experiments showed that increasing Transformer depth improved spatial overlap, whereas more complex convolutional configurations increased both performance and training time. Overall, the findings indicate that biomimetic integration of hierarchical convolutional processing and attention-based global representation provides a promising strategy for brain MRI segmentation and registration.