Must New Drugs Be Superior to Those Already Available? The Role of Noninferiority Clinical Trials

While visionary work was conducted earlier in the 20th century by the statistician Sir Ronald Fisher,1, 2 the continued development and widespread adoption of the randomized clinical trial as the “gold standard” for demonstrating a drug's efficacy was driven by the 1962 Kefauver-Harris Amendments to the US Federal Food, Drug, and Cosmetics Act of 1938.3 Initial attention focused on the superiority trial, in which our intent is to look for compelling evidence that an investigational (new) drug demonstrates superior efficacy compared with a control drug (either a placebo or an active control that may be the standard of care at the time). Other study designs have now been developed and widely adopted, one of which is the noninferiority trial. Given the global need for better hypertension control4, 5 and the use of noninferiority trials in this therapeutic domain,6-9 this paper describes their fundamental components and overall approach. While a new drug's efficacy is clearly of great interest to us, other factors influence its suitability for therapeutic use and the degree to which it will be successful in the marketplace if approved. These include its tolerability, its safety profile (and, hence, in combination with its efficacy, the overall benefit-risk profile), and its cost, which will be influential in the extent to which its use may be reimbursed by payers, and therefore the degree to which it may be prescribed. Consider an example in which the side-effect profile of a new drug is dramatically better than that of an existing (reference) drug for the same indication. In this scenario we may be prepared to accept a certain degree of reduced efficacy since, despite this reduced efficacy, the overall benefit-risk balance of the new drug may be considerably more favorable. The question of interest therefore becomes: What degree of reduced efficacy are we prepared to accept, given the other advantages of the drug? If the new drug has only this degree, or less than this degree, of reduced efficacy, we will declare it to be noninferior to the reference drug. Expressed slightly differently, the goal of a noninferiority trial is to look for compelling evidence that a new drug's efficacy is, in the worst case, “acceptably worse” than (ie, is noninferior to) that of the reference drug, typically the existing gold-standard treatment for the disease of interest. Establishing that a new drug is noninferior to a reference drug has two key components: defining noninferiority using clinical judgment, and then using an appropriate statistical test. Imagine a scenario where a new drug has such a better safety profile compared with the reference drug that we are prepared to accept a certain decrease in efficacy. The degree of reduced efficacy we are prepared to accept is a clinical question. The study team must choose this value, which is called the noninferiority margin. This decision must be made before the trial is conducted, and the noninferiority margin included in the study protocol. The noninferiority margin is therefore part of the study design. Given several attractive advantages of a new antihypertensive agent, let's hypothetically say that a decrease in mean efficacy of 3.0 mm Hg in systolic blood pressure (SBP) is acceptable. Our choice means that, in our study design, the noninferiority margin will have a value of 3.0 mm Hg. Unlike superiority trials, for which P values are employed to determine whether a new drug's efficacy is statistically significantly superior to that of the comparator drug, P values are not typically used in noninferiority trials. Establishing noninferiority is based on the use of confidence intervals (CIs).10 International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use Guideline E911 states that, in this case, we should use a one-sided CI. The term one-sided CI conveys that our attention only need be paid to one limit, upper or lower, and the other one can therefore be left unspecified. In our explanations and our two hypothetical case studies, attention falls on the upper limit, or bound, of the commonly chosen one-sided 95% CI. The reason that we do not need the lower bound has two parts: (1) if the new drug's efficacy actually turns out to be greater than that of the reference drug, that would be a perfectly acceptable outcome that would still lead to the drug being declared noninferior to the reference drug; and (2) this is a true statement no matter how much better its efficacy, and we therefore do not need a specific bound to be placed below the noninferiority margin. The calculated CI will tell us whether we will reject the null hypothesis in favor of the research hypothesis, or whether we will fail to reject the null hypothesis. The location of the upper bound of the 95% CI that is placed around the treatment effect, calculated as mean reduction in SBP for the new drug minus mean reduction in SBP for the reference drug, determines whether the null hypothesis is rejected. If the value of the upper bound is <3.0 mm Hg (the value of the noninferiority margin), we reject the null hypothesis and declare the new drug to be noninferior. If the upper bound is 3.0 mm Hg or greater, we fail to reject the null hypothesis and the new drug is not declared to be noninferior, ie, it is inferior. We should note here that some authors report two-sided CIs. This is also a legitimate strategy, and two-sided 90% CIs and two-sided 95% CIs can be seen in the literature. Mathematically, the upper limit of a two-sided 90% CI is equivalent to the upper limit of a one-sided 95% CI. Similarly, the upper limit of a two-sided 95% CI is equivalent to the upper limit of a one-sided 97.5% CI. Our operational conduct of the trial would be exactly the same as it would be if the design being employed were a superiority design instead of a noninferiority design. That is, we would randomize participants to either the new drug treatment group or the reference drug treatment group and then use optimal quality research methodology to collect the data. In this case, the data addressing the trial's primary endpoint are the baseline and end-of-study SBP levels for each participant, and hence the reduction in SBP for each participant during their participation in the trial. The mean reduction in SBP for each treatment group would then be calculated. However, the statistical analysis and interpretation strategy now employed differs from that used in a superiority trial: Once the treatment effect point estimate has been determined, attention moves directly to calculation of the upper bound of the 95% CI to be placed around the treatment effect point estimate. Imagine the results from a noninferiority trial involving New Drug A. The mean SBP response for the reference drug was a reduction of 10.0 mm Hg, which we will write algebraically as −10.0 mm Hg. The mean SBP response for New Drug A was a reduction of 8.5 mm Hg, written as −8.5 mm Hg. The treatment effect point estimate, calculated as the mean new drug response minus the mean reference drug response, is therefore 1.5 mm Hg (the calculation is −8.5 minus −10.0, which is equivalent to −8.5 plus 10.0, which is 1.5). That is, New Drug A lowers SBP on average 1.5 mm Hg less than the reference drug. We now calculate the upper bound of the 95% CI and place it around the treatment effect point estimate. Say that this value is 1.9 mm Hg. This is the value of key importance. We see that it is less than the value of our noninferiority margin, ie, 3.0 mm Hg. We therefore reject the null hypothesis in favor of the research hypothesis, and declare New Drug A to be noninferior to the reference drug. That is, the degree of reduced efficacy associated with the new drug is considered to be “acceptably worse” compared with the reference drug since the new drug offers other salient advantages. We can now use this statement to make our decision. Since the value of key importance, the upper bound of the one-sided 95% CI placed around the treatment effect point estimate, lies below the noninferiority margin of 3.0 mm Hg, we declare New Drug A to be noninferior to the reference drug. Imagine now the results from a noninferiority trial involving New Drug B. As for the trial involving New Drug A, the mean SBP response for the reference drug was a reduction of 10.0 mm Hg, written algebraically as −10.0 mm Hg. The mean SBP response for New Drug B was a reduction of 7.2 mm Hg, written as −7.2 mm Hg. The treatment effect point estimate, again calculated as the mean new drug response minus the mean reference drug response, is therefore 2.8 mm Hg (the calculation is −7.2 minus −10.0, which is equivalent to −7.2 plus 10.0, which is 2.8). That is, New Drug B lowers SBP on average 2.8 mm Hg less than the reference drug. We now calculate the upper bound of the 95% CI and place it around the treatment effect point estimate. In this case, say that the value is 3.4 mm Hg. Again, this is the value of key importance. We see that it is greater than the value of our noninferiority margin, ie, 3.0 mm Hg. We therefore fail to reject the null hypothesis, and hence we declare that New Drug B is inferior to the reference drug. That is, the degree of reduced efficacy associated with the new drug is considered to be “unacceptably worse” compared with the reference drug despite the other salient advantages of the new drug: on balance, those advantages do not outweigh its inferior efficacy. We can now use this statement to make our decision. Since the value of key importance, the upper bound of the one-sided 95% CI placed around the treatment effect point estimate, lies above the noninferiority margin of 3.0 mm Hg, we do not declare New Drug B to be noninferior to the reference drug, ie, it is inferior. Note that we make this decision even though the treatment effect point estimate itself lies below the noninferiority margin. Having presented two hypothetical examples, one where a drug was found to be noninferior and one where a drug was not found to be noninferior, consider the following real data that were published in this journal by Matsuoka and colleagues.7 The authors investigated whether a new antihypertensive drug, TY-0201 (TY 8 mg), which is delivered via a transdermal patch would be found to be noninferior to an existing antihypertensive drug, bisoprolol fumarate oral formulation (BO 5 mg). As the authors noted, “Antihypertensive drugs in the form of transdermal patches are considered to be highly clinically significant in terms of stable control of BP during a 24-hour period” since they sustain a constant plasma concentration. They are also suitable for patients who experience difficulty in swallowing. The primary endpoint was a change from baseline in office diastolic blood pressure (DBP) at week 8. A noninferiority margin of 3.3 mm Hg was chosen. Since the value of key importance, the upper bound of 1.4 mm Hg, lay below their noninferiority margin of 3.3 mm Hg, the authors declared TY 8 mg to be noninferior to BO 5 mg. It is important to emphasize here that since the upper bound of 1.4 mm Hg was greater than zero, the authors could not conclude that TY 8 mg was superior in efficacy to BO 5 mg. That said, the trial did indeed demonstrate that TY 8 mg was noninferior to BO 5 mg, leading the authors to conclude that, given its advantages as noted earlier, they expected it “to serve as a new treatment approach for hypertensive patients.”7 A good understanding of noninferiority trials is informative on how and why a new drug that has been declared noninferior to a drug already on the market is now available for therapeutic use. This understanding can be very helpful to physicians when deciding, on a case-by-case basis, whether the new drug is appropriate for their patients. The authors report no specific funding in relation to the preparation of this manuscript. No editorial support was used.