Key result
EMA-distillation ViT adaptation improves carotid plaque segmentation by ~0.05 Dice over ImageNet initialization.
Why the study?
Large numbers of carotid ultrasound images are available without task-specific annotations, and it was uncertain whether self-supervised adaptation to unlabelled carotid B-mode ultrasound improves Vision Transformer representations relative to ImageNet initialization.
Does self-supervised adaptation using EMA-based latent self-distillation on unlabelled carotid B-mode ultrasound images improve Vision Transformer representation transfer for classification and plaque segmentation compared to ImageNet initialization?
Does self-supervised adaptation using EMA-based latent self-distillation on unlabelled carotid B-mode ultrasound images improve Vision Transformer representation transfer for classification and plaque segmentation compared to ImageNet initialization?
Domain-specific self-supervised learning via EMA-based latent self-distillation on unlabelled carotid ultrasound images improves Vision Transformer representation transfer for downstream classification and plaque segmentation tasks compared to standard ImageNet initialization.
Answers are grounded in this paper and cite their sources.
Self-distilled ViTs may enhance carotid ultrasound segmentation; leaves open clinical validation and broader adoption.
Background: Large numbers of carotid ultrasound images are available without task-specific annotations. This study investigated whether self-supervised adaptation to unlabelled carotid B-mode ultrasound improves Vision Transformer (ViT) representations relative to ImageNet initialization. Methods: An ImageNet-pretrained ViT-S/16 was adapted using 2,840 unlabelled carotid B-mode images from CUBS and Orvile. We used an I-JEPA-inspired EMA-based latent self-distillation approach and a MAE-style masked-reconstruction approach for comparison. The learned representations were evaluated in three experiments. Experiment A assessed frozen representations using a CUBS grayscale proxy-classification task at 25%, 50% and 100% label fractions. Experiment B1 assessed frozen transfer to plaque segmentation on the independent Ar-PlaqSegm1 dataset, while Experiment B2 compared ImageNet and EMA-distillation initialization after supervised fine-tuning. All downstream experiments used three random seeds. Results: EMA-distillation achieved the highest mean AUROC at all label fractions: 0.866, 0.876 and 0.882, compared with 0.844, 0.858 and 0.876 for ImageNet. For frozen segmentation, mean Dice was 0.409 ± 0.022 versus 0.356 ± 0.033 for ImageNet, with a paired image-level difference of +0.0526 (95% bootstrap CI, +0.0033 to +0.1010). After fine-tuning, mean Dice increased to 0.467 ± 0.016, compared with 0.368 ± 0.028 for ImageNet. Conclusion: EMA-distillation adaptation improved transfer relative to ImageNet initialization with both frozen encoders and supervised fine-tuning. The results support further study of domain-specific self-supervised learning for carotid ultrasound. This study evaluates representation learning and transfer, not clinical diagnostic performance.
No takes yet. Share an insight, caveat, or question.
Cher Lim (2026) studied Carotid plaque / carotid ultrasound imaging (n=2,840). EMA-based latent self-distillation (I-JEPA-inspired) Vision Transformer (ViT-S/16) adaptation vs. ImageNet-pretrained ViT-S/16 initialization (and MAE-style masked-reconstruction) was evaluated on Frozen transfer to carotid plaque segmentation (mean Dice score) on the Ar-PlaqSegm1 dataset (MD +0.0526, 95% CI +0.0033 to +0.1010). EMA-distillation adaptation of ViT improved frozen carotid plaque segmentation vs ImageNet initialization (mean Dice 0.409 vs 0.356; difference +0.0526, 95% CI 0.0033-0.1010).
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: