A note on the magnitude of hazard ratios

Conclusions from the interesting article by Borate et al1 regarding nonbiological factors affecting survival in younger patients with acute myeloid leukemia would be strengthened if the following 3 methodological issues were addressed. First, the analyses used significance testing as the sole determinant of the relevance of a predictor on survival, which, in relatively large samples, as was the case in their study, opens up the possibility of confusing statistical “significance” with clinical importance or consequence (ie, the magnitude or large sample size fallacy).2, 3 What effect size (ie, hazard ratio [HR]) do the authors consider clinically relevant, regardless of significance tests? This would be a qualitative assessment using subject matter expertise, but perhaps one way to provide a frame of reference for the question is by converting the HR into a measure of effect size for which, at least in some disciplines, there is some agreement regarding magnitudes representing small (not trivial, but possibly inconsequential), medium (likely consequential), and large (very likely consequential) effects, such as the standardized mean difference d4 (small, 0.2; medium, 0.5; and large, 0.8). For example, a small d is the difference in height between 15-year-old and 16-year-old girls,4 whereas a large d is the difference in height between 13-year-old and 18-year-old girls.4 Odds ratios can be transformed into standardized mean differences5 (in the log odds scale), which are used in meta-analysis. HRs are more complex than odds ratios and a transformation into a standardized mean difference may require some simplifying assumptions. Under exponentially distributed survival time, a simple model6 with constant and therefore proportional hazards, the mean, median, and variance of the survival time are functions of the hazard rate λ only (mean, 1/λ; median, ln(2)/λ; and variance, 1/λ2). Because the variance of this survival time depends on the hazard rate, computing a standardized mean difference in survival times between 2 groups with different hazard rates is unclear because there is no common standard deviation. However, taking the natural log of the exponentially distributed survival time stabilizes the variance,7 yet the mean and median log survival times remain functions of the hazard rate: mean=−ln(λ)−γ, in which γ is the Euler-Mascheroni constant (γ≈0.5772); median=−ln(λ)+ln(ln(2)); and variance=π2/6. With that common variance, a standardized mean difference in the log survival time can be defined. After some algebra, d in the natural log of the exponentially distributed survival time reduces to , which would also be the standardized difference in medians of the log survival time. Arguably, a threshold for the likelihood of relevance for d can be established at 0.35, the midpoint between the “small” and “medium” effect sizes. This threshold in terms of an HR would equal 1.57, assuming an exponentially distributed survival time. Even though the actual survival data might not follow an exponential model, effect sizes using this assumption may be used as simple references. Small, medium, and large HRs comparing 2 groups would be approximately 1.3, 1.9, and 2.8, respectively. Following a similar argument for a continuous predictor (using the measure of effect r4 instead of d), small, medium, and large HRs for a standard deviation increase in the predictor would be 1.14, 1.47, and 1.9, respectively, with a threshold for the likelihood of relevance at a HR of 1.3 for a standard deviation increase in the predictor. Another issue in the article by Borate et al1 related to significance testing is the fact that that predictor selection for the Cox models was conducted with backward selection via P values. Although trying to remove unnecessary predictors from a model may be reasonable, removal based on sequential P values renders inferences based on the final P values invalid8 without a proper correction, because of multiple testing. A proper correction for this procedure is not trivial. Criterion-based approaches such as Bayes information criterion,9 while being no modeling silver bullets, may be preferred over testing-based variable selection.9 Alternative nonlinear models such as survival trees10, 11 also can help provide robustness of conclusions if they select a similar set of predictors compared with the Cox proportional hazards approach. Finally, there was no mention of whether the proportional hazard assumptions of the Cox models were examined. If strong deviations from the assumptions are present, the models can be fixed by including time-by-covariate interactions, but the interpretation becomes complex. Again, other modeling techniques such as a survival trees that do not assume proportional hazards may be used as an alternative. It would be interesting to determine whether the conclusions of the analyses change after interpreting the magnitudes of the effects, and under different variable selection or modeling approaches. No specific funding was disclosed. The author made no disclosures. Andres Azuero, PhD Department of Family, Community and Health Systems University of Alabama at Birmingham Birmingham, Alabama