What question did this study set out to answer?

June 7, 2026Open Access

Benchmarking Deep Vision Architectures: From Controlled Datasets to Real-World Clinical Validation

RARaghda Essam AliDigital Science (United States)NENoha Ahmed Saad El-Dien MMMagi Hossam Eldin Mahfouz

Key Points

This study aims to benchmark deep learning architectures for automated stroke classification from brain CT images and assess their clinical applicability.
Introduced NeuroCT-Bench, a benchmark for evaluating models on the Brain Stroke CT Image dataset with 1,551 normal and 950 stroke slices.
Compared five CNNs (VGG16, ResNet50, DenseNet121, EfficientNetB0, MobileNetV2) and four Transformers (Swin Transformer, ViT, DeiT, PVT-Small) under identical protocols.
Conducted an ablation study on a CNN–Transformer gated fusion strategy to evaluate its effect on performance.
PVT-Small and Swin Transformer achieved near-perfect internal validation ROC-AUCs of 99.96% and 99.98%, respectively.
EfficientNetB0 showed the highest external dataset accuracy at 76.92% and ROC-AUC of 88.0%, while Transformer models experienced significant performance drops.
Adaptive fusion improved robustness and generalization over individual models, indicating the benefits of combined approaches.

Abstract

Stroke is a leading cause of mortality and long-term disability, making rapid and reliable detection from non-contrast computed tomography (CT) scans essential for timely clinical intervention. This study introduces NeuroCT-Bench, a unified and reproducible benchmark for evaluating deep learning architectures for automated stroke classification from brain CT images. The benchmark systematically compares five Convolutional Neural Networks (VGG16, ResNet50, DenseNet121, EfficientNetB0, MobileNetV2) and four Vision Transformers (Swin Transformer, ViT, DeiT, PVT-Small) under identical preprocessing, augmentation, and evaluation protocols using the Brain Stroke CT Image dataset from Kaggle, comprising 1,551 normal and 950 stroke slices. Internal validation using a deterministic 80/20 stratified split (seed = 42) demonstrated near-perfect performance for Transformer-based models, with PVT-Small and Swin Transformer achieving ROC-AUCs of 99.96% and 99.98%, respectively, while DenseNet121, VGG16, and EfficientNetB0 achieved strong CNN baselines with ROC-AUCs of 99.93%, 99.67%, and 99.05%. To evaluate robustness under real-world domain shift, the top-performing models were further assessed on an external patient-level clinical dataset containing 530 CT studies. EfficientNetB0 demonstrated the strongest generalization capability (accuracy: 76.92%, precision: 83.85%, ROC-AUC = 88.0%), whereas high-capacity Transformer models exhibited substantially larger performance degradation (ROC-AUCs of 75.0% for PVT-Small and 67.0% for Swin Transformer). These findings highlight the discrepancy between curated public datasets and heterogeneous clinical imaging conditions, emphasizing that high internal performance does not necessarily guarantee clinical robustness. In addition, an ablation study was conducted to evaluate a lightweight CNN–Transformer gated fusion strategy. Results demonstrate that adaptive fusion improves robustness and generalization compared with individual CNN or Transformer models and static feature concatenation. Overall, NeuroCT-Bench provides a transparent and reproducible framework for evaluating deep learning models for stroke analysis and supports future development of clinically deployable hybrid CNN–Transformer systems.

Ask AI

Helpful

Bookmark

View Full Paper