Analysis and Properties of the Generalized Total Least Squares Problem AX B When Some or All Columns in A are Subject to Error

The Total Least Squares (TLS) method has been devised as a more global fitting technique than the ordinary least squares technique for solving overdetermined sets of linear equations AX B when errors occur in all data. This method, introduced into numerical analysis by Golub and Van Loan, is strongly based on the Singular Value Decomposition (SVD). If the errors in the measurements A and B are uncorrelated with zero mean and equal variance, TLS is able to compute a strongly consistent estimate of the true solution of the corresponding unperturbed set A₀ X = B₀. In the statistical literature, these coefficients are called the parameters of a classical errors-in-variables model. In this paper, the TLS problem, as well as the TLS computations, are generalized in order to maintain consistency of the parameter estimates in a general errors-in-variables model; i. e. , some of the columns of A may be known exactly and the covariance matrix of the errors in the rows of the remaining data matrix may be arbitrary but positive semidefinite and known up to a factor of proportionality. Here, a computationally efficient and numerically reliable Generalized TLS algorithm GTLS, based on the Generalized SVD (GSVD), is developed. Additionally, the equivalence between the GTLS solution and alternative expressions of consistent estimators, described in the literature, is proven. These relations allow the main statistical properties of the GTLS solution to be deduced. In particular, the connections between the GTLS method and commonly used methods in linear regression analysis and system identification are pointed out. It is concluded that under mild conditions the GTLS solution is a consistent estimate of the true parameters of any general multivariate errors- in-variables model in which all or some subsets of variables are observed with errors.