The demand for High-Performance Computing (HPC) has been growing since the rise of AI in the last few years. Supercomputers are in demand more than ever across many fields for running generative models, bitcoin mining, gaming services, and other computing applications that are increasingly run in the cloud. Scheduling in HPC has always been one of the most interesting research areas, especially in the 2000s, when grid computing emerged as a promising new technology in the domain. However, research in this area has been in the shadows for a while as the grid has matured into cloud-based systems in practice. Scheduling for HPC seemed to be a saturated research area, while new research trends have taken off in all directions. The recent big leap in AI models has revived this domain for two main reasons. Many classic, heuristic, and metaheuristic scheduling algorithms can be reconceptualized by integrating them with machine learning-based algorithms or frameworks, likely leading to better performance than the classical models used previously. Additionally, the urgent need for supercomputing for running the AI models that are used widely and daily, almost for every task, urges the area of scheduling algorithms itself. This paper proposes two novel AI models. The first model is based on the hurdle framework. In contrast, the second model is based on a stacking classifier followed by a stacking regression for two real datasets, using three performance metrics: wait time, slowdown, and response time, compared with the Random Forest solo model. The Hurdle model achieved the best performance on zero-inflated targets, while the Two-Stacking Stage showed better temporal robustness. When zero inflation is absent, simpler models can be more effective.
Ömer Dakkak (Tue,) studied this question.