Key result
Selective PEFT matches or exceeds full fine-tuning for ECG foundation models while improving generalizability.
Parameter-efficient fine-tuning of ECG foundation models offers a computationally efficient alternative to full fine-tuning that enhances model generalizability and reduces overfitting, particularly on out-of-domain data.
Foundation models (FMs) are large-scale models pretrained on extensive datasets to learn general representations that can be adapted to multiple tasks through fine-tuning. FMs are gaining traction in the electrocardiogram (ECG) analysis field, also thanks to their ability to effectively address downstream tasks for which less data is available. However, the choice of the adaptation strategy is often underestimated, usually relying only on full fine-tuning, ignoring the risk of overfitting due to the high model’s complexity and the limited dataset size. In this study, we propose selective parameter-efficient fine-tuning (PEFT) as an alternative to full fine-tuning to increase model generalizability and reduce overfitting. Specifically, we examine partial fine-tuning, BitFit, LayerNorm tuning, and linear probing, with a controlled protocol on Self-DANA foundation model. We evaluate the generalizability of the fine-tuned models on unseen in-domain and out-of-domain data, and with different downstream tasks and dataset characteristics. We demonstrate that when a proper selective PEFT strategy is chosen for the given scenario, it can outperform full fine-tuning not only in resource efficiency, but also in enhancing model generalization, especially on out-of-domain data.
No takes yet. Share an insight, caveat, or question.
Monachino et al. (2026) studied Arrhythmia and cardiac diagnosis (ECG analysis). Selective parameter-efficient fine-tuning (PEFT) vs. Full fine-tuning and supervised training from scratch was evaluated on AUROC for downstream ECG classification tasks (in-domain and out-of-domain). Selective parameter-efficient fine-tuning (PEFT) of an ECG foundation model achieved comparable or superior AUROC compared to full fine-tuning, while reducing overfitting and enhancing generalizability on out-of-domain data.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: