Answers are grounded in this paper and cite their sources.
Modern GPU clusters support Deep Learning training (DLT) jobs in a distributed manner. Job scheduling is the key to improve the training performance, resource utilization and fairness across users. Different training jobs may require various objectives and demands in terms of completion time. How to efficiently satisfy all these requirements is not extensively studied.
No takes yet. Share an insight, caveat, or question.
Gao et al. (2021) studied this question.