Energy-aware resource allocation for data analysis workflows in heterogeneous cluster environments

Scientific workflows automate complex data analysis processes and are increasinglyemployed to handle large-scale datasets. Scientific communitiesleverage heterogeneous computing clusters, consisting of interconnected resourceswith diverse computational capabilities, to execute their large-scaledata analysis workflows due to privacy and economic factors. Consequently,they often impose high computational demands and lead to significant energyconsumption. Designing an effective energy-aware scheduling approachis therefore crucial for heterogeneous computing systems, as it enables highprocessing efficiency while minimizing overall energy usage. However, achievingboth generality in workflow structures and solvability of the correspondingoptimization models remains a major challenge in workflow scheduling.In this paper, we introduce a novel mathematical approach that models thenon-overlapping execution of tasks successfully by representing disjoint timeintervals as linear constraints. By applying this technique, we formulate amixed-integer linear programming model that can be solved within a reasonabletime, providing exact optimal solutions for workflows of arbitrary structure.We analyzed the trade-off between energy consumption and makespan.As part of the post-optimality analysis, sensitivity analysis and relaxationof optimal energy consumption in pursuit of makespan improvement are explored.Results indicated that our proposed approach outperforms relatedmethods.