Different analyses estimate different parameters of the effect of erythropoietin stimulating agents on survival in end stage renal disease: a comparison of payment policy analysis, instrumental variables, and multiple imputation of potential outcomes

OBJECTIVE: To compare the assumptions and estimands across three approaches to estimate the effect of erythropoietin-stimulating agents (ESAs) on mortality. STUDY DESIGN AND SETTING: Using data from the Renal Management Information System, we conducted two analyses using a change to bundled payment that, we hypothesized, mimicked random assignment to ESA (pre-post, difference-in-difference, and instrumental variable analyses). A third analysis was based on multiply imputing potential outcomes using propensity scores. RESULTS: There were 311,087 recipients of ESAs and 13,095 non-recipients. In the pre-post comparison, we identified no clear relationship between bundled payment (measured by calendar time) and the incidence of death within 6 months (risk difference -1.5%; 95% confidence interval [CI] -7.0%, 4.0%). In the instrumental variable analysis, the risk of mortality was similar among ESA recipients (risk difference -0.9%; 95% CI -2.1, 0.3). In the multiple imputation analysis, we observed a 4.2% (95% CI 3.4%, 4.9%) absolute reduction in mortality risk with the use of ESAs, but closer to the null for patients with baseline hematocrit level >36%. CONCLUSION: Methods emanating from different disciplines often rely on different assumptions but can be informative about a similar causal contrast. The implications of these distinct approaches are discussed.