The Mahalanobis distance and its relationship to principal component scores

The Mahalanobis distance is one of the most common measures in chemometrics, or indeed multivariate statistics. It can be used to determine whether a sample is an outlier, whether a process is in control or whether a sample is a member of a group or not. In the article on the chi-squared and multinormal distributions, we discussed the role of the Mahalanobis distance in the chi-squared distribution, although it is used in many other situations 1. Although many people use this distance in chemometrics, it is poorly understood, especially its relationship to principal components analysis (PCA), another very common technique. “The squared Mahalanobis distance is equal to the sum of squares of the scores of all non-zero standardised principal components.” The Mahalanobis distance was first proposed by the Indian statistician P. C. Mahanobis in 1936 2. The 1930s were important years for the development of multivariate concepts, primarily in biology, economics, and psychology, with many famous names such as R. A. Fisher and H. Hotelling active during these years and communicating together. For traditional univariate statistics, it is usual to calculate the number of standard deviations an observation is from the centre of a dataset and use this value to determine various statistics about it. Extending to multivariate situation, Mahalanobis proposed a distance Δ from the centre of the data. A ' represents a transpose; that is, the rows and columns are interchanged. In later articles, we will brush up on matrix algebra for those that are unfamiliar. When there is only one variable, this simplifies to , where ν is the variance or ν, giving where s is the standard deviation of the data. Hence, the Mahalanobis distance for one variable is the same as the number of standard deviations; an observation is away from the mean. For more than one variable, the Mahalanobis distance can be visualised as the distance of a point from the centre of a dataset, onto an ellipse (if there are two variables) whose main direction is that of the data, as illustrated in Figure 1. The ellipse drawn is that of equal Mahalanobis distance so any point lying on that ellipse will be equally far from the centroid. The axes of the ellipse are scaled so that they are both of the same length for points at equal Mahalanobis distance from the centre. There is however an alternative way of visualising the Mahalanobis distance, and that is by transforming the original data, as illustrated in Figure 2 for a dataset characterised by two variables. There are of course many other diverse reasons for using the Mahalanobis distance measure besides the chi-squared criterion, and we will be using this measure and comparing to other distances in different contexts in future articles. The Mahalanobis distance has a number of interesting properties. For a given dataset (or training set), the sum of squares of the Mahalanobis distance of all observations, or rows in a data matrix, usually equals the product of the number of variables times the number of observations. So if the sample size is 50, and there are three variables, the sum of the 50 squared Mahalanobis distances will usually be 150. This is because the sum of squares each normalised principal component equals the sample size. This can be verified numerically. Of course, there are other and computationally somewhat simpler ways of checking this in packages such as Matlab, but for the purpose of this article, we primarily illustrate the steps in Excel. In fact, Excel is a good way of learning and consequently understanding methods because one can see all the numerical steps and check sums as one goes on. There are many outstanding Matlab programmers in chemometrics, but not every one of them would realise the connection between the Mahalanobis distance and principal components analysis, despite using both on a regular basis, possibly because it is possible to skip numerical stages when using Matlab. There are several on-line resources that can help mainly with calculations and further insight. Dwinnel 3 presents a Matlab-oriented article. Wicklin's article 4 although written by SAS institute, requires no knowledge of SAS to understand and is a helpful discussion.