On the information-based measure of covariance complexity and its application to the evaluation of multivariate linear models

This paper introduces a new information-theoretic measure of complexity called ICOMP as a decision rule for model selection and evaluation for multivariate linear models. The development of ICOMP is based on the generalization and utilization of the covariance complexity index of van Emden (1971) in estimation of the multivariate linear model. ICOMP is motivated by Akaike's (1973) Information Criterion (AIC), but it is a different procedure than AIC. In linear or nonlinear statistical models ICOMP uses an information-based characterization of: (i) the covariance matrix properties of the parameter estimates of a model starting from their finite sampling distributions, and (ii) the complexity of the inverse-Fisher information matrix (i-FIM) as a new criterion of achievable accuracy of the model As a result, it provides a trade-off between the accuracy of the parameter estimates and the interaction of the residuals of a model via the measure of complexity of their respective covariances. It controls the risks of both insufficient and overparameterized models, and incorporates the assumption of dependence and the independence of the residuals in one criterion function. A model with minimum ICOMP is chosen to be the best model among all possible competing alternative models. ICOMP relieves the researcher of any need to consider the parameter dimension of a model explicitly. A real numerical example is shown in subset selection of variables in multivariate regression analysis to demonstrate the utility and versatility of the new approach.