An Algorithm for Restricted Least Squares Regression

A commonly occurring problem in statistics is that of minimizing a least squares expression subject to side constraints. Here a simple iterative algorithm is presented and shown to converge to the desired solution. Several examples are presented, including finding the closest concave (convex) function to a set of points and other general quadratic programming problems. The dual problem to the basic problem is also discussed and a solution for it is given in terms of the algorithm. Finally, extensions to expressions other than least squares are given.