What is Not What in Statistics

Some 40 years ago, Harold Hotelling pointed out that the statistical textbooks of that period were written largely by non-mathematicians. Those books were full of misconceptions, and were rather uniformly unaware of the new and dramatic development of the mathematical discipline of statistical inference. They did not take advantage of the sharpened logic for making decisions about populations on the basis of sample statistics, including the improved logic of estimation and of hypothesis testing. The situation was slowly remedied as more mathematical statisticians began to issue textbooks, until today the pendulum may have swung too far. In some quarters, the symbols of inference rather than the substance may have taken over. This appears to be especially true in the social sciences with which I am most acquainted, and to which this paper is largely (but not exclusively) addressed. For example, referees and editors of some journals insist on decorating tables of various kinds of data with stars and double stars, and on presenting lists of standard errors, despite the fact that the implied probabilities for significance or confidence are quite erroneous from the point of view of statistical inference (see Problems 3 and 1 below).