Comparison of activation functions for optimizing deep learning models solving QSM-based dipole inversion

Deploying deep learning models for quantitative susceptibility mapping is driven by optimizing hyper-parameters and using suitable architectures. We investigated the impact of activation functions on network model training. ELU-, leaky ReLU- and ReLU-models with 16 and 32 initial channels were tested for solving dipole inversion on synthetic susceptibility data. All models showed convergence after completing 100 training epochs. However, the 16-channel-ELU-model achieved low losses after only 20 training epochs and showed similar reconstruction performance to the 32-channel-ELU-model. Using the ELU activation allows the use of smaller network models resulting in fewer memory requirements and less training time.