The essential role of balance tests in propensity‐matched observational studies: Comments on ‘A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003’ by Peter Austin, Statistics in Medicine

Peter Austin has made an exacting, timely and eye-opening review of uses of propensity-score matching in medical research. Its Section 2.1 argues that the reports of propensity-matched analyses should include descriptive assessments of matched treatment-control differences on baseline variables. When propensity matching on covariates including BMI, for example, one should report the difference between matched cohorts' mean BMIs, perhaps after inverse scaling by the pooled s.d. of BMIs prior to matching. The recommendation is a good one: matched differences on prognostic variables, and on variables that track selection into treatment, speak to the credibility of subsequent matched outcome analyses; and although the basic promise of propensity matching is that it should lessen such differences, the extent of the reduction varies greatly from case to case. Furthermore, since successful propensity matches or subclassifications enable comparisons similar to those which randomization would have given-in terms of observed covariates, at least [1], and should those covariates jointly suffice to remove confounding, then also in terms of outcomes [2]-it follows that balance is the basic mark of success of a propensity adjustment.