U is for Unease: Reasons for Mistrusting Overlap Measures for Reporting Clinical Trials

In recent years a number of authors have promoted approaches to measuring the results of clinical trials that depend on the degree of overlap between the distribution of results from the treatment and the control group. In their enthusiasm for such measures, however, many have overlooked a particular problem, namely that while they seem to overcome a certain arbitrary element in measuring the shift between groups in a given trial, they still depend very strongly on the distribution of the results from trial to trial.