Rank-based analysis of the desirability of outcome ranking methodology: confidence intervals and hypothesis testing

Randomized clinical trials are the gold standard for evaluating the benefits and harms of interventions, though often fail to provide the practical evidence to inform medical decision-making. The Desirability Of Outcome Ranking (DOOR) is a pragmatic paradigm for the design, monitoring, analysis, reporting and interpretation of clinical trials and other research studies based on patient-centric benefit:risk evaluation, developed to address this issue and advance clinical trial science. The DOOR uses an ordinal composite outcome to analyze patients by comparing the experiences of trial participants in different treatment groups by the desirability of the overall patient response. One analysis in the DOOR paradigm compares the outcome between interventions using rank-based methodologies and estimating the DOOR probability, i.e. the probability that a participant assigned to one intervention has a more desirable outcome than a participant assigned to the other intervention. In this article, we review statistical methods for constructing a confidence interval and testing a hypothesis for the DOOR probability and discuss improved methods. We evaluate the performance of the methods by simulation. We illustrate the use of the methods using examples. We provide practical guidance on the use of confidence intervals and hypothesis tests associated with the DOOR probability.