We present an enhanced version of the row-based randomized block-Kaczmarz method to solve a linear system of equations. This improvement makes use of a regularization during block updates in the solution, and a dynamic proposal distribution based on the current residue and effective orthogonality between blocks. This improved method provides significant gains in solving high-condition number linear systems that are either sparse, or dense least-squares problems that are significantly over/under determined. Considering the poor generalizability of preconditioners for such problems, it can also serve as a pre-solver for other iterative numerical methods when required, and as an inner iteration in certain types of GMRES solvers for linear systems.
Kar et al. (Thu,) studied this question.