Answers are grounded in this paper and cite their sources.
The regulatory agencies in the United States (Food and Drug Administration) and European Union (European Medicines Agency) have recently released draft guidance documents on the general topic of multiplicity in clinical trials 1, 2. The EMA's draft guideline serves as a revision of the Point to Consider document published 15 years ago 3. The FDA's guidance document emphasizes multiplicity considerations related to the analysis of multiple end points. However, the general principles presented in the guidance are broadly applicable as they are relevant in the context of clinical trials with several dose-control comparisons or several pre-defined patient populations. The draft guidance documents review the fundamental principles of multiple comparisons, for example, the concept of strong familywise error rate control, and they provide a comprehensive coverage of issues arising in confirmatory clinical trials with multiple objectives. Numerous practical considerations and even technical details are presented. For example, the FDA guidance introduces traditional multiple testing procedures and also gives a detailed discussion of advanced procedures that have been applied to complex problems with several multiplicity components. The information and recommendations presented in the guidance documents will encourage drug developers to learn more about novel approaches to performing multiplicity adjustments and to judiciously apply them in confirmatory clinical trials. It is worth pointing out a few multiplicity-related topics that are not explicitly discussed in the draft guidance documents. As stated above, the guidelines focus on confirmatory trials, and it is important to recognize the value of addressing multiplicity in early-phase trials, especially in Phase II trials. Multiplicity control at this stage helps trial sponsors avoid incorrect drug development decisions based on spurious findings from smaller trial data sets. To give an example, a key feature of the popular MCP-Mod algorithm is that it relies on a multiplicity adjustment to arrive at a reliable dose-finding strategy 4. A broad application of this and similar methods will help improve the probability of identifying effective doses for the confirmatory stage and may ultimately increase the overall success rate in Phase III development. In addition, global statistical tests are often applied in Phase II trials for conditions that exhibit a complex etiology. The efficacy profile of an experimental treatment is typically described using several efficacy end points. A formal multiplicity correction will lower statistical power and will be counterproductive in this setting. Instead of performing a univariate assessment of each individual end point, clinical trial sponsors should consider a global test that explicitly accounts for the relationships among the end points 5. The global testing approach capitalizes on a synergistic nature of the end points and provides a substantial improvement over standard univariate analysis strategies. The general topic of multiplicity discussed in the FDA and EMA draft guidelines has attracted much attention in the clinical trial community due to the increasing complexity of modern Phase III clinical trials that commonly employ multiple instruments to fully characterize the efficacy and safety of experimental treatments. To provide an overview of common multiplicity problems in confirmatory clinical trials and strategies for addressing multiplicity, it is instructive to distinguish between two settings, or decision-making scenarios, that can be labeled as traditional and advanced multiplicity problems. In a traditional setting, the trial's sponsor needs to deal with a single multiplicity component or, in other words, a single source of multiplicity. Examples of trials with a single multiplicity component include trials with one primary end point evaluated in two or more patient populations, for example, in the general trial population and pre-defined subsets based on relevant baseline patient characteristics 6. Multiplicity in this setting may also be induced by the assessment of treatment effects at several doses compared to a common control 7. Multiple testing procedures utilized in traditional problems focus on controlling the overall type I error rate (or familywise error rate) in the strong sense at a pre-defined alpha level, for example, one-sided α = 0.025. Advanced settings with several sources of multiplicity are often encountered in trials with hierarchically ordered families of clinical objectives. For instance, multiple primary and secondary end points are commonly employed in the development of treatments for conditions that manifest themselves in multidimensional ways. In this case, the primary end point or end points support the primary clinical objective of a trial. Secondary end points provide additional clinical information and are regarded as less important than the primary end points; for more information on different types of secondary end points, see D'Agostino 8. The treatment effect on pre-specified end points may be evaluated in several dosing arms with the analysis of several end points defining one source of multiplicity and the assessment of several dose-control contrasts defining another source of multiplicity. The corresponding tests are grouped into families to reflect their importance; for example, the primary end point tests are placed in the primary family and are carried out first followed by the tests from the other family. Using the brexpiprazole development program in schizophrenia as an example 9, the efficacy profile of two doses of the experimental treatment was evaluated using the change from baseline in the Positive and Negative Syndrome Scale (PANSS) total score as the primary end point and change from baseline in the Clinical Global Impressions Severity Scale (CGI-S) score as the key secondary end point. The natural approach is to set up two families that include the dose-control tests on the primary end point and dose-control tests on the key secondary end point, respectively. Similar hierarchical families of clinical objectives also arise in multi-population trials with several end points; see, for example, the BELLE-2 trial where the treatment effects on progression-free survival (PFS) and overall survival (OS) were studied in the general population and a subgroup of patients with certain gene mutations 10. To support an efficient formulation of regulatory claims based on secondary outcome measures or, more generally, secondary trial objectives, it is critical for the trial's sponsor to ensure that the type I error rate is controlled with respect to all sources of multiplicity. Multiple testing procedures, known as gatekeeping procedures, are commonly used in settings with two or more sources of multiplicity. The innovative approaches to formulating analysis strategies in modern clinical trials have inspired numerous researchers to begin actively working on the development of novel statistical methods for challenging problems with several sources multiplicity. As a direct result, there has been an enormous amount of progress in the general area of multiplicity research over the past 15 years. Dozens of new approaches to handling multiplicity in a variety of different settings have been developed. These approaches support valid decision making with respect to complex sets of clinical objectives in confirmatory trials, including the traditional and advanced settings defined above. Easy-to-implement algorithms have been constructed, and software tools, including open-source tools, have been built to facilitate the implementation of new approaches in real-life trials. To learn more about the recent advances in multiplicity-related research, readers are encouraged to consult the review papers and tutorials published in Statistics in Medicine, including Dmitrienko, D'Agostino, and Huque 11, Dmitrienko and D'Agostino 12, and Alosh, Bretz, and Huque 13. These novel methods for dealing with multiplicity have been successfully applied to numerous late-stage clinical trials. For instance, powerful gatekeeping procedures were constructed to preserve the overall type I error rate in multiplicity problems with three hierarchical families in the lurasidone Phase III trials 14, 15. The families were defined based on a single primary and two key secondary end points (first source of multiplicity) and two or three dose-placebo comparisons (second source of multiplicity). Relevant features of the underlying multiplicity problems were incorporated into the decision rules; for example, positive correlations among the dose-placebo tests were taken into account to improve the procedure's efficiency. To give other examples, gatekeeping procedures have been developed to control the overall type I error rate with respect to several sources of multiplicity and enrich product labels in development programs for the treatment of insomnia 16 or type 2 diabetes mellitus 17. For a comprehensive review of multiplicity adjustment strategies in recent analgesic clinical trials, see Gewandter et al. 18. Industry-wide guidelines stress that it is critical to prospectively define strategies for addressing multiplicity induced by multiple outcome measures and other sources. For example, the recommendations released by the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials 19 state that “to decrease the probability of a Type I error … in pain clinical trials, the use of gatekeeping procedures and other methods that correct for multiple analyses is recommended when a single primary endpoint does not adequately reflect the overall benefits of treatment”. Of course, with the power and flexibility of recently developed multiplicity adjustment methods comes great responsibility. The number of available options may be overwhelming and, as pointed out by O'Neill 20 in a more general context, “the many statistical methods and approaches that have been suggested in the literature in recent years now deserve to be digested and placed in the context of how they can be best used in clinical trials.” Clinical trial sponsors are encouraged to follow a principled approach to ensure the relevance and desired performance of multiple testing procedures in Phase III trials 21. First, logical relationships among the individual clinical objectives, for example, relationships among the primary and secondary end points 22 or hierarchical relationships among the doses 23, ought to be incorporated. This step results in a set of clinically relevant candidate procedures. Furthermore, to identify a procedure with desirable performance characteristics, it is recommended to conduct a systematic quantitative evaluation of the candidate procedures. This evaluation would be aimed at selecting the most efficient and robust multiple testing procedure under the trial-specific assumptions. The Clinical Scenario Evaluation (CSE) approach 24 provides a useful framework for performing these evaluations. In addition, CSE-based optimization methods have been developed and successfully applied to select optimal multiplicity adjustment methods in common settings 25. This issue of Statistics in Medicine includes four papers that provide commentaries aimed at deepening and complementing the general discussion presented in the FDA and EMA draft guidance documents. The first three papers 26-28 examine a broad set of issues arising in the analysis of multiple clinical end points, and the last paper 29 discusses multiplicity issues pertinent to subgroup analysis. Sankoh, Li, and D'Agostino 26 present general considerations related to the use of composite and multi-component end points in clinical trials. Guidelines for evaluating the individual components of composite end points, delineation of “soft” and “hard” components, and multiplicity corrections used in the analysis of composite end points are reviewed in this paper. Analysis of hierarchically ordered end points, for example, primary and secondary end points, as well as composite end points, is discussed in Snapinn 27. The paper examines several specific cases, including non-inferiority and superiority testing with several end points, procedures with a fixed testing sequence, and analysis of end point components. Chuang-Stein and Li 28 provide a detailed discussion of “reverse multiplicity problems” where the overall outcome in a trial is positive if a significant treatment effect is established simultaneously on all pre-defined end points, known as co-primary end points. The recommended approach to test each end point at a nominal level (one-sided α = 0.025) is known to result in power loss, especially in clinical trials with large number of co-primary end points, and the authors go over several alternative approaches to analyzing co-primary end points. Finally, Dmitrienko, Millen, and Lipkovich 29 review multiplicity-related considerations in the context of subgroup analysis. The authors define two main classes of subgroup evaluation problems in confirmatory clinical trials (exploratory and confirmatory subgroup analysis) and introduce statistical methods for multiplicity correction within each class. As a quick note, multiplicity issues inherent in exploratory subgroup exploration are not explicitly discussed in the FDA and EMA draft guidance documents; however, this topic plays a central role in the recently released EMA guideline on the investigation of subgroups in confirmatory clinical trials 30. The choice of a multiplicity adjustment procedure has a major impact on the overall conclusions in a Phase III trial or a development program. When selecting among available multiple testing strategies, it is of paramount importance to critically evaluate all relevant clinical and statistical information and ensure that the selected multiplicity adjustment strategy is aligned with the regulatory guidelines. We hope that the draft guidance documents and additional considerations presented in the discussion papers will help facilitate the decision-making process in trials with complex clinical objectives.
No takes yet. Share an insight, caveat, or question.
Dmitrienko et al. (2017) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: