Leveraging GPU Batching for Scalable Nonlinear Programming through Massive Lagrangian Decomposition

.We present the implementation of a trust-region Newton algorithm ExaTron for bound-constrained nonlinear programming problems, fully running on multiple graphics processing units (GPUs). Without data transfers between a CPU and GPU, our implementation has achieved the elimination of a major performance bottleneck under a memory-bound situation, particularly when solving many small problems in batch. We discuss the design principles and implementation details for our kernel function and core operations. Different design choices are justified by numerical experiments. By using the application of distributed control of alternating current optimal power flow, where a large problem is decomposed into many smaller nonlinear programs using a Lagrangian approach, we demonstrate computational performance of ExaTron on the Summit supercomputer at Oak Ridge National Laboratory. Our numerical results show the linear scaling with respect to the batch size and the number of GPUs and more than 35 times speedup on 6 GPUs than on 40 CPU cores available on a single node.Reproducibility of computational results.This paper has been awarded the "SIAM Reproducibility Badge: Code and data available" as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/exanauts/ExaTron.jl and in the supplementary materials (ExaTron.jl-main.zip [101KB]).KeywordsGPU optimization solversecond-order optimization methoddecompositionMSC codes65K0590C0690C3090-0490-08