Answers are grounded in this paper and cite their sources.
In the previous article, we introduced how to calculate mean sums of squares (MSS) for errors (or variability) in data. When centred, these MSS are analogous to variances. The fundamentals of ANOVA involve comparing the size of these MSSs to determine whether an effect, factor, or model is significant. Statistical significance can then be used to answer scientific questions, for example, “does pH have an influence over the yield of a reaction?” and “is there a quadratic relationship between hardness and composition and how important is it?.” Usually, a number is translated into a probability. For example, we might like to say “we are over 99% certain that it is important to use a specific and rather expensive additive to ensure that the taste of a type of drink is acceptable to the market.” The mean sum of squares can be presented in a more comprehensible manner, and turned into probabilities or indicators of significance. The previous article showed how to calculate MSS for different situations, but we ended up with a jumble of numbers, and no clear way of interpreting them. We could certainly deduce, for example, that the lack-of-fit was small, and therefore, for the proposed model, our data were fitted better than the instrument or, analytical procedure, could determine. In this article, we will show how to present the MSS in a more comprehensible manner and how to turn these numbers into probabilities or indicators of significance. We will focus on estimating whether a mathematical model in its entirety is significant, rather than individual terms in a model, which will be discussed in later articles. An interesting observation is that the coefficients for the models of Tables 1 and 2 are the same, but they differ for the squared terms for the model of Table 3, as presented in Table 4. This is because the squared terms are not centred in the design matrix. A little mentioned way of overcoming this inconsistency is to centre the squared terms in the design (or model) matrix. Most traditional statistical texts do not mention this problem, because most applications of ANOVA are to models just with linear and interaction terms. Under such circumstances, it does not matter if the data are centred before building the model, or afterwards. Most traditional statistics assumes that background data are drawn from a multinormal distribution. The sum of squares of a multinormal distribution can be approximated using an F distribution. Fisher's work, which is an important basis of modern applied statistics, introduced the concept of the null hypothesis. We may hypothesize, for example, that the lack-of-fit has no significance. Were this to be so, we would be confident that our model fits the data well. Usually, if a p value is .10 or less, we can reject the null hypothesis. A value of p = .05, for example, suggests that we would expect a variance ratio of the given value or greater only 5% of the time if the null hypothesis was correct. ANOVA tables are sometimes produced with p values. The lower the p value is for a given ratio, the more reliably we can reject the null hypothesis that a particular source or model or parameter is not significant. In most cases, the F (or variance) ratio involves comparing one source of error or variation against the replicate (or analytical) error. There is an important caveat here that many processes are not normally distributed, so the F value should be interpreted with caution. If it is very low (eg, p = .01 or less), then we are probably safe rejecting the null hypothesis because the ratio is so extreme. If, however, it is much higher (eg, p = .20), then interpretation crucially depends on whether the data can be well approximated by a normal distribution. In many practical cases, such as metabolomics, this is definitely not so. A good rule of thumb is to accept very extreme p values (.01 or less) as a convincing rejection of the null hypothesis, but where the p values are somewhat higher, to perform other tests as well to back up the hypothesis. Both the F test via ANOVA and t test can be used further to estimate the significance of individual terms in the model, as will be discussed in later articles. It though is important not to get too excited by ANOVA, which is often overused, particularly because data are rarely normally distributed. Very extreme values of lack-of-fit (or low p values) are still good guidance, and ANOVA tables do provide us with a good way of summarizing sources of variability. They can be used in a wide variety of contexts. Many chemists routinely come across only certain types of design. This is because variability can usually be controlled. A reaction performed under well-specified conditions should give the same yield whether in a laboratory in China or the United States or Brazil. In fact, if this were not so, there would be little point, for example, developing a process in the UK, and then expecting it to work well in a factory in Singapore. However, a biologist or clinician or consumer survey agency might have quite different perspectives. If we test a drug on a population in London, will it be effective on a population in Bangkok? Each population will have different genetics, different diets, and different environments. We cannot just select a group of people by height, weight, age, and gender and expect the same result. Most of chemometrics is applied to processes that take place in controlled environments. We can select the same size flasks, set the same temperature and pH in both London and Bangkok, and providing our chemicals are in the same concentrations and in the same solvent systems in both places, achieve near enough the same results. Not all chemometrics is applied to chemistry, but a central feature is that most relates to controlled laboratory-based experimentation even if in medicine or biology, rather than field studies. So, the emphasis in areas such as medicine or population biology or psychology is less on estimating parameters from multilinear models and more on concepts such as blocking and randomization. ANOVA is therefore often introduced in different ways in applied statistics according to the sort of problems encountered. A statistician may spend a year learning about the different types of ANOVA and only occasionally encounter multilinear models. To a chemist, ANOVA is almost synonymous with regression and so a first understanding has been introduced via model building.
No takes yet. Share an insight, caveat, or question.
Richard G. Brereton (2018) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: