Systematic Review of Hybrid Vision Transformer Architectures for Radiological Image Analysis

Background Vision Transformer (ViT) and Convolutional Neural Networks (CNNs) each possess distinct strengths in medical imaging: ViT excels in capturing long-range dependencies through self-attention, while CNNs are adept at extracting local features via spatial convolution filters. However, ViT may struggle with detailed local spatial information, critical for tasks like anomaly detection in medical imaging, while shallow CNNs may not effectively abstract global context. Objective This study aims to explore and evaluate hybrid architectures that integrate ViT and CNN to lever-age their complementary strengths for enhanced performance in medical vision tasks, such as segmentation, classification, and prediction. Methods Following PRISMA guidelines, a systematic review was conducted on 28 articles published between 2020 and 2023. These articles proposed hybrid ViT-CNN architectures specifically for medical imaging tasks in radiology. The review focused on analyzing architectural variations, merging strategies between ViT and CNN, innovative applications of ViT, and efficiency metrics including parameters, inference time (GFlops), and performance benchmarks. Results The review identified that integrating ViT and CNN can mitigate the limitations of each architecture, offering comprehensive solutions that combine global context understanding with precise local feature extraction. We benchmarked the articles based on architectural variations, merging strategies, innovative uses of ViT, and efficiency metrics (number of parameters, inference time(GFlops), performance). Conclusion By synthesizing current literature, this review defines fundamental concepts of hybrid vision transformers and highlights emerging trends in the field. It provides a clear direction for future research aimed at optimizing the integration of ViT and CNN for effective utilization in medical imaging, contributing to advancements in diagnostic accuracy and image analysis. Summary Statement We performed systematic review of hybrid vision transformer architecture using PRISMA guideline and perfromed through meta-analysis to benchmark the architectures. ACM Reference Format Ji Woong Kim, Aisha Urooj Khan, and Imon Banerjee. 2018. Systematic Review of Hybrid Vision Transformer Architectures for Radiological Image Analysis. J. ACM 37, 4, Article 111 (August 2018), 16 pages. https://doi.org/XXXXXXX.XXXXXXX