Jaca: Model-training and cluster feature based job scheduling for distributed deep learning cluster

Deep Learning Training (DLT) jobs have become the major workload in cloud GPU clusters. As an increasing variety of distinctly different DLT jobs are trained simultaneously on clusters and these jobs adopt different parallelism strategies, the challenge lies in how to schedule these jobs to efficiently utilize cluster resources considering their communication patterns. In this paper, we introduce Jaca, a job scheduler aiming to reduce Job Completion Time (JCT) by selecting the job with the highest affinity—a metric we define to reflect how efficiently a job can run on the current cluster—based on jobs’ communication patterns and current cluster loads. To efficiently calculate the affinity within a reasonable time, we group the machines of the cluster, sample and prune placement orders to reduce the exploded placements plan space. This approach comprehensively considers the impact on the cluster, ultimately scheduling jobs along with their placement. Evaluation on a 12-GPU cluster demonstrates that Jaca improves the average JCT by 1.40 × ∼ 2.82 × . The large-scale trace-based simulation further demonstrates that Jaca improves average JCT by up to 1.45 × ∼ 2.40 × compared with the traditional job schedulers like RL-Hybrid, Yarn, Tiresias, and Gandiva.