Probability, P values, and statistical significance: instructions for use by surgeons

There is possibly no single term more misused, or idea misunderstood, than probability values, or P values, in the scientific literature. The most commonly admitted definition of the P value is the probability of obtaining results at least as extreme as those observed of a statistical hypothesis test (plausibility of data from a sample (your results)), assuming the null hypothesis (statistical hypothesis which proposes that no difference exists between 2 groups in a given series)1. The concept has been attributed to the famous English mathematician and statistician Ronald Fisher, when he was confronted with the assertion of a well known British lady that she could detect whether milk, or hot water, was poured first into her cup of tea. Astounded at the idea that the lady could do so by taste alone, he devised an experiment in which he set up eight cups of tea for her to taste (half with water poured first, half with milk poured first), served in random order, the famous ‘tea test’ (not to be confused with the t test derived by William Gosset under the pen name of Student). When she correctly identified all eight cups, he calculated that the probability of this accomplishment would be 1 in 70 (approximately 1.43 per cent or P = 0.0114)2.