Generalized Rules for Calculating the Magnitude Of an Effect in Factorial and Repeated Measures ANOVA Designs

Many present day researchers are using the phrases approaching significance, highly significant, and dramatically significant in an attempt to describe the magnitude, and therefore the inferred importance, of effects in comparative experiments. The use of a particular phrase is dependent on the relative difference between the standard alpha probability value, .05 or .01, and the obtained or looked up probability value associated with the observed t or F for an effect. For an example, an F ratio with a reported p < .10 is reported as approaching significance whereas an F with a reported p < .0001 is reported as dramatically significant. Such reporting of obtained probability values has received severe criticism (Hays, 1963; Cohen, 1969; Bakan, 1966) indicating that the size of the obtained t or F, together with its corresponding p value, depends directly on the size of the sample and does not necessarily reflect the magnitude or importance of the treatment effect.