Reflective block Kaczmarz algorithms for least squares

In Steinerberger, Q. Appl. Math., 79:3, 419-429, 2021 and Shao, SIAM J. Matrix Anal. Appl. 44(1), 212-239, 2023, two new types of Kaczmarz algorithms, which share some similarities, for consistent linear systems were proposed. These two algorithms not only compete with many previous Kaczmarz algorithms but, more importantly, reveal some interesting new geometric properties of solutions to linear systems that are not obvious from the standard viewpoint of the Kaczmarz algorithm. In this paper, we comprehensively study these two algorithms. First, we theoretically analyse the algorithms given in Steinerberger, Q. Appl. Math., 79:3, 419-429, 2021 for solving least squares. Second, we extend the two algorithms to block versions and provide their theoretical convergence rates. Our numerical experiments also verify the efficiency of these algorithms. Third, as a theoretical complement, we address some key questions left unanswered in Shao, SIAM J. Matrix Anal. Appl. 44(1), 212-239, 2023.