A predictive offline–online hybrid approach to DNN training energy optimization

With the rapid development of artificial intelligence, GPUs are playing an increasingly vital role in data centers. However, the high-performance computing capabilities of GPUs also bring about significant energy consumption issues. Although existing methods attempt to improve energy efficiency by optimizing GPUs or workloads, they often lack comprehensive considerations. To address this challenge, this paper proposes an energy-aware strategy that jointly optimizes GPU power capping and workload batch size to improve both GPU energy efficiency and workload performance. The strategy consists of two components: offline and online optimization. In the offline phase, we introduce a combined KNN-XGBoost prediction model to forecast power consumption and performance under different configuration combinations and use the Energy-Delay Sum (EDS) metric to identify optimal configurations. In the online phase, linear searches are conducted around these offline-optimal configurations, and the resulting power, performance, and energy efficiency metrics are evaluated to determine the most effective configuration. Experimental evaluations show that, compared to default and offline-only optimization strategies, our proposed method significantly reduces GPU energy consumption, particularly for deep neural network workloads. Specifically, it reduces average total GPU energy consumption by 13.2%, decreases workload running time by 14.1%, and optimizes the EDS metric by 13.8%.