The Role of Optimizing Loss Functions in Transfer Learning to Address Data Scarcity

Motivation: Data scarcity in medical imaging presents substantial challenges for training complex deep learning models aimed at solving inverse problems like field-to-susceptibility inversion. Goal(s): This study investigates the effectiveness of pre-training and transfer learning, focusing specifically on loss function optimization during transfer learning. Approach: An Adaptive 3D U-Net was pre-trained on synthetic data and subsequently fine-tuned on in-vivo brain datasets using various loss functions. The models were evaluated on 32 brain datasets. Results: Using complex loss functions penalizing intensity and structural deviations during transfer learning resulted in improved susceptibility map accuracy, whereas their application during pre-training did not yield better outcomes. Impact: This study demonstrated the potential of optimizing transfer learning to adapt pre-trained models, even from different training settings, to new target-specific data, highlighting the great potential of cross-domain knowledge transfer and fine-tuning in addressing data scarcity.