CLSP: Linear Algebra Foundations of a Modular Two-Step Convex Optimization-Based Estimator for Ill-Posed Problems

This paper develops the linear-algebraic foundations of the Convex Least Squares Programming (CLSP) estimator and constructs its modular two-step convex optimization framework, capable of addressing ill-posed and underdetermined problems. After reformulating a problem in its canonical form, \(A⁽ʳ⁾ z⁽ʳ⁾ = b\), Step~1 yields an iterated (if \(r > 1\)) minimum-norm least-squares estimate \({z}⁽ʳ⁾ = (AZ⁽ʳ⁾)^ b\) on a constrained subspace defined by a symmetric idempotent \(Z\) (reducing to the Moore-Penrose pseudoinverse when \(Z = I\)). The optional Step~2 corrects \({z}⁽ʳ⁾\) by solving a convex program, which penalizes deviations using a Lasso/Ridge/Elastic net-inspired scheme parameterized by \(α ∈ [0,1]\) and yields \({z}^*\). The second step guarantees a unique solution for \(α ∈ (0,1]\) and coincides with the Minimum-Norm BLUE (MNBLUE) when \(α=1\). This paper also proposes an analysis of numerical stability and CLSP-specific goodness-of-fit statistics, such as partial \(R^2\), normalized RMSE (NRMSE), Monte Carlo \(t\)-tests for the mean of NRMSE, and condition-number-based confidence bands. The three special CLSP problem cases are then tested in a 50,000-iteration Monte Carlo experiment and on simulated numerical examples. The estimator has a wide range of applications, including interpolating input-output tables and structural matrices.