The Convergence of the Ben-Israel Iteration for Nonlinear Least Squares Problems

Ben-Israel [1] proposed a method for the solution of the nonlinear least squares problem min x ∈ D \| F(x) \|₂ where F:D ⊂ Rⁿ → Rᵐ. This procedure takes the form xk + 1 = xₖ - F’(xₖ)^ + F(xₖ) where F’(xₖ)^ + denotes the Moore-Penrose generalized inverse of the Fréchet derivative of F. We give a general convergence theorem for the method based on Lyapunov stability theory for ordinary difference equations. In the case where there is a connected set of solution points, it is often of interest to determine the minimum norm least squares solution. We show that the Ben-Israel iteration has no predisposition toward the minimum norm solution, but that any limit point of the sequence generated by the Ben-Israel iteration is a least squares solution.