Optimizing Heterogeneous High-Performance Computing Resource Allocation for Optimal Performance in Advanced Seismic Imaging Algorithms

Optimizing resource allocation for seismic imaging workloads in High-Performance Computing (HPC) environments, particularly in heterogeneous infrastructures with diverse architectures, is crucial for achieving optimal computational efficiency in the oil and gas industry. As seismic imaging algorithms continue to grow in sophistication, they exhibit diverse resource requirements, making it essential to understand and accommodate these varying demands to ensure optimal resource utilization. This paper presents a data-driven resource allocation strategy that leverages application-level behavior metrics to optimize workload placement and resource utilization in heterogeneous HPC environments. The methodology is centered around defining two key metrics: Application CPU Ratio (ACPR) and Size CPU Ratio (SCR), which enable accurate identification of the applications’ resource consumption patterns to guide decisions on optimal resource allocations. By identifying the optimal number of cores required for each application, this approach maximizes system throughput and eliminates idle resources. The proposed optimization method has been successfully applied to production HPC environment, without requiring any modifications to the underlying infrastructure, resulting in significant improvements in system efficiency, with a substantial 100X decrease in queue wait times, enhancing overall resource utilization. The methodology's effectiveness underscores its potential to transform resource management in various scientific and engineering domains.