Answers are grounded in this paper and cite their sources.
Demonstrates enhanced numerical accuracy and speed of SVD in GPU systems, suggesting improved performance in matrix computations.
With the advancement of GPU architecture, matrix computation engines such as NVIDIA Tensor Cores now support double-precision (FP64) General matrix multiplications (GEMMs) with the same efficiency as single-precision (FP32) GEMMs. However, the adoption of this enhanced FP64 capability remains limited, primarily restricted to applications that involve multiple FP64 BLAS3 operations. Singular Value Decomposition (SVD), a fundamental decomposition in numerical linear algebra with numerous applications, can greatly benefit from exploiting this hardware feature. In this paper, for FP32 SVD, we propose a novel algorithm, FP64 precision eigenvalue decomposition (EVD) based SVD, specifically designed to leverage the latest GPU architectural features. We provide a theoretical analysis demonstrating the feasibility of our approach on emerging GPU architectures and evaluate it from both accuracy and performance perspectives. Moreover, for FP64 SVD, we introduce a double-blocking band reduction technique combined with a GPU-based bulge chasing algorithm to further accelerate the overall SVD process. Experimental results show that, for FP32 SVD, our EVD-based SVD implementation achieves higher numerical accuracy and delivers speedups of up to 6.1 × on H100 and 5.0 × on A100 over the state-of-the-art cuSOLVER SVD solver. In the case of FP64 SVD, our method also achieves 4.9 × and 4.8 × speedups on H100 and A100, respectively. These results highlight the potential of our approach as a highly efficient and accurate solution for SVD on modern GPU platforms.
No takes yet. Share an insight, caveat, or question.
Wang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: