Integrating convolutional neural networks with multi-head self-attention for accurate brain tumour classification in MRI images

Brain tumour classification from MRI images is a critical task in medical diagnosis; however, it remains challenging due to dispersed visual patterns and subtle inter-class variations. To address this issue, this study presents a hybrid deep-learning model based on the interaction of the spatial feature extraction characteristics of Convolutional Neural Networks (CNNs) and the global context representation of Multi-Head Self-Attention (MHSA). Initially, a hierarchical CNN backbone is used to process input images into feature maps used to create spatial representations. These representations are then transformed into a set of tokens and routed through a self-attention architecture to capture long-range dependencies among images. This arrangement enables effective local-global integration of the learned representations. The learned representations are aggregated using global average pooling and fed into fully connected layers for classification into four categories: glioma, meningioma, pituitary tumour, and no tumour. The results from the experiment using 7,023 MRI images demonstrate that the hybrid deep learning architecture provided a peak validation accuracy of 91.8%, F1 score of 0.92%, outperforming several baseline approaches.