Machine learning (ML) applications, which have become quite common tools for many high energy physics (HEP) applications, benefit significantly from GPU resources. To fulfill the rapidly increasing demand for GPU resources, the efficient utilization of GPU clusters is vital. The Karlsruhe Institute of Technology (KIT) provides a GPU cluster, accessible via a traditional batch system as well as via grid compute elements. Because the exact hardware needs of such applications heavily depend on the ML hyperparameters, a flexible resource setup is necessary to utilize the available resources as efficiently as possible. Therefore, the multi- instance GPU (MIG) feature of the NVIDIA A100 GPUs was studied. Several neural network (NN) training scenarios performed on the GPU cluster at KIT are discussed to illustrate the setup that has been used and possible performance gains.
Voigtlaender et al. (Thu,) studied this question.