What question did this study set out to answer?

This research aims to improve medical image classification and segmentation by optimizing architecture choices and adaptation methods.

May 17, 2026Open Access

From architecture to adaptation : empirical recommendations for medical image classification and segmentation

Key Points

This research aims to improve medical image classification and segmentation by optimizing architecture choices and adaptation methods.
Controlled comparison of CNNs, Vision Transformers, and hybrid architectures across chest X-ray and dermatology imaging tasks.
Investigation of parameter-efficient fine-tuning strategies using LoRA and IA³ on foundation models.
Synthesis into a hybrid architecture combining CNN and LoRA-adapted transformer for improved performance.
Self-supervised CNNs achieve highest average accuracy/AUC across six MedMNIST+ tasks, demonstrating importance of architectural biases.
LoRA fine-tuning matches full fine-tuning performance while updating fewer parameters and achieves competitive segmentation results.
Hybrid architecture provides strong segmentation performance and substantial reduction in trainable parameters.

Abstract

Deep learning has significantly advanced medical image analysis, yet selecting appropriate architectures, pretraining strategies, and adaptation methods remains challenging. This thesis addresses these design questions through three interconnected studies spanning classification and segmentation in chest X-ray and dermatology imaging. The first study presents a controlled comparison of convolutional neural networks (CNNs), Vision Transformers (ViTs), and hybrid architectures across multiple medical image classification benchmarks. Models are matched by parameter count and trained under identical protocols, with additional evaluation of medical foundation models under linear probing. Results show that self-supervised CNNs achieve the highest average accuracy/Area Under the Curve (AUC) across six MedMNIST+ tasks, that architectural inductive biases often matter more than model scale, and that foundation models can underperform smaller fine-tuned models when domain alignment is limited. The second study investigates parameter-efficient fine-tuning (PEFT) of the MedMAE foundation model using Low-Rank Adaptation (LoRA) and Infused Adapter by Inhibiting and Amplifying Inner Activations (IA³). Across classification and segmentation tasks, LoRA achieves performance comparable to full fine-tuning while updating only a small fraction of model parameters, and matches full fine-tuning for segmentation. IA³ provides greater compression at modest performance cost. The third study synthesizes these findings into a hybrid architecture that prepends a trainable CNN stem to a LoRA-adapted transformer backbone. Systematic evaluation demonstrates that this design achieves the strongest segmentation results and competitive classification performance while substantially reducing the number of trainable parameters. Together, these studies demonstrate that effective medical image analysis requires alignment between architecture, pretraining objective, and adaptation strategy. The thesis provides a systematic benchmark, empirical validation of PEFT for medical foundation models, and a principled hybrid design that balances performance and efficiency.

Perguntar à IA

Bookmark

View Full Paper