The chi squared and multinormal distributions

A series of measurements may fall into a multivariate normal distribution but not a chi squared distribution, unless they are independent. The first description of the chi squared (or χ2) distribution was by the German mathematician Frederick R. Helmert in the late 19th century1, although it was not clearly formulated in statistics until Karl Pearson developed the chi squared test in 1900, who independently rediscovered the distribution 2. The modern name and statistical terminology arises primarily from Pearson's work. When there is one variable measured, the chi squared distribution can be very simply related to the normal distribution. Chi squared is simply the square of the distance to the mean. If the underlying observations are normally distributed, we obtain the chi squared distribution. Therefore, for example, if data are standardised (that is measurements are divided by the standard deviation after subtracting the mean), the proportion of observations less than four units of chi squared from the mean is the same as the proportion of observations, within two normalised units either side of the mean. This is illustrated in Figure 1. The chi squared distribution becomes important in chemometrics when more than one variable is measured. For the normal distribution, we can look at how far a datapoint is from the mean, and this may be used to determine whether it is an outlier or whether it belongs to a predefined group. If the underlying distribution is assumed to be normal, a probability can be attached as discussed in The normal distribution 3. We would like to perform a similar calculation in cases where there is more than one variable measured. Once the number of variables exceeds 1, there is no specific positive or negative direction to the mean or centre of the data, so instead, a squared distance must be calculated. We can see that when one variable is recorded, if it is distributed normally, its square is distributed according to chi squared. Before we return to chi squared using more than one variable, it is necessary first to understand the multivariate normal distribution. In the definition used above, it is not necessary for the variables to be independent or uncorrelated (we will discuss the difference between the two in a later article). Figure 2 illustrates two bivariate multinormal distributions. In the top one, both variables are independent, but in the bottom one, they are not. It so happens that the bottom one is simply the rotation of the top one. However, it is important to understand that both distributions satisfy the conditions of multivariate normality. For the normal distribution, we can determine whether an observation is likely to belong to a predefined population, according to how many standard deviations it is away from the mean. If the probability is 5%, then we only expect to find an observation that far from the mean one 1 time in 20. Likewise for a multinormal distribution, the distance from the mean or centre of the measurements can be used to determine the likelihood of membership of a predefined population. Because distance in more than one dimension does not have a predefined sign, we have to use squared distances. Can we use the squared distance from the mean of a dataset to tell us how likely an observation is to be part of a predefined population, analogous to the one variable case? Hence, a series of measurements may fall into a multivariate normal distribution but not a chi squared distribution, unless they are independent. So, how can we use chi squared to determine whether an observation belongs to a predefined population by looking at the distance from the centre? When only one variable is measured, it is of course simple. A new measurement of distance comes into play, the Mahalanobis distance. This will be discussed in detail the next article, but it breaks the correlation and also puts both variables on the same scale. The Mahalanobis distance when there is more than one variable can be thought analogous to the standard deviation. Figure 3 is of the Mahalanobis distance of 2 (or a squared distance of 4) units from the centre of a bivariate normal distribution. We have used the chi squared distribution with 2 degrees of freedom. As more variables are measured, the shape of the chi squared distribution changes. Figure 4 represents chi squared for several different values of k or number of variables or dimensionality of the variable space or degrees of freedom (sometimes denoted by ν). Remember that the definition of the χ2 statistic is the squared Mahalanobis distance and that this distribution is only obeyed if the underlying distribution is multinormal (so, one could calculate the squared Mahalonobis distances of samples that are not multinormal, and they would not fall into a chi squared distribution). Remember that the shape of the multinormal distribution, for example, as characterised by the sum of the individual measurements for each observation, is the same as the normal distribution no matter how many variables are measured, unlike the chi squared distribution. In many traditional books, the chi squared distribution is often presented in tabular form. A typical table is presented in Table 1, although there are other ways of presenting it. The rows represent the number of degrees of freedom or variables in our case and the columns the cumulative probability. The columns are often called critical p-values. There are numerous resources that describe further mathematical properties of the χ2 distribution, two online ones which are referenced here 4, 5.