NII Technical Report (NII-2004-006E)：Preconditioned GMRES Methods for Least Squares Problems

For least squares problems of minimizing || b - A x ||₂ where A is a large sparse m x n (m >= n) matrix, the common method is to apply the conjugate gradient method to the normal equation AT A x = AT b. However, the condition number of AT A is square of that of A, and convergence becomes problematic for severely ill-conditioned problems even with preconditioning. In this paper, we propose two methods for applying the GMRES method to the least squares problem by using a n x m matrix B. We give the necessary and sufficient condition that B should satisfy in order that the proposed methods give a least squares solution. Then, for implementations for B, we propose an incomplete QR decomposition IMGS (l). Numerical experiments show that the simplest case l=0, which is equivalent to B= (diag (AT A) ) ^ (-1) AT, gives best results, and converges faster than previous methods for severely ill-conditioned problems.