Q-GMRES: Quantized Data Transmission Optimization for Distributed GMRES Algorithm

The generalized minimal residual (GMRES) algorithm is one of the key iterative algorithms for solving large-scale sparse linear systems, which finds extensive application in scientific and engineering computations. Although distributed GMRES can handle large-scale problems, data transmission between multiple nodes emerges as a primary bottleneck. Given the inefficient utilization of finite bits in the IEEE 754 format and the potential impact of low-precision floating-point representations on the convergence of iterative algorithms, this paper proposes a quantization-based method (Q-GMRES) to compress the data transmitted between nodes. Our approach not only reduces the amount of data transmission, but also uses the limited bit-width representation to achieve high accuracy. In this paper, we explore uniform and non-uniform quantization techniques and propose inner-outer mixed-precision and vector segmentation methods to improve residual accuracy. Experimental results demonstrate that the proposed communication compression method, which combines INT16 quantization with vector segmentation, achieves speedups of up to 2.19x and 2.49x on four and eight processes, respectively, while maintaining convergence accuracy comparable to GMRES-FP64.