Trial Design for Mold-Active Agents: Time to Break the Mold--Aspergillosis in Neutropenic Adults

Invasive aspergillosis causes high morbidity and mortality; thus, there is a need for more effective therapies. However, regulatory approval of new agents is based on the results of randomized clinical trials (RCTs), which take years and huge amounts of money to complete [1]. Novel strategies to evaluate effective therapies for aspergillosis, quickly and efficiently, are urgently needed. In this issue of Clinical Infectious Diseases, Cornely et al. [2] report the results of an RCT of primary therapy for mold infections (97% of which were aspergillosis), asking whether escalation of the dose of liposomal amphotericin B improves outcome. Patients were randomized to receive liposomal amphotericin B at 3 or 10 mg/kg per day for 14 days, followed by receipt of 3 mg/kg per day. The primary end point was favorable outcome (i.e., complete or partial response at end of therapy), and the median duration of therapy was 14–15 days. Survival and drug-related adverse events were secondary end points. Most patients had pulmonary aspergillosis and were neutropenic. No significant differences in response between the treatment groups were detected, although recipients of the 10 mg/kg daily dose experienced higher rates of nephrotoxicity and hypokalemia. Uncontrolled malignancy and allogeneic stem cell transplantation were associated with poorer survival. These findings [2] add to the evidence that escalation of the dose of antifungals (in this case, liposomal amphotericin B) may increase toxicity without necessarily improving outcome, at least in patients with aspergillosis. In addition, these findings demonstrate the fallacy of allegations—raised in the lay press [3]—that underdosing of deoxycholate–amphotericin B [4] and liposomal amphotericin B [5] increased the number of deaths in RCTs of empirical antifungal therapy. An evidence-based rebuttal of these unsubstantiated allegations also raised concerns about dose-dependent toxicity associated with amphotericin B [6]. The study by Cornely et al [2]—the first to investigate antifungal therapy dosing with a large sample size—is noteworthy for several of its strengths. Rigorous methodology.The study was double-blind and had oversight by data safety and data review committees. Homogeneous population.The study enrolled neutropenic adults with hematological malignancies. Detailed analysis of prognostic factors.Disease status, neutropenia, and allogeneic stem cell transplantation were analyzed. Efficient design.The study enrolled 339 patients in 19 months, underscoring the symbiotic relationship between patients and clinical trials. Such an efficient design benefits the study (and future patients), but more importantly, it also helps enrollees, because every effort is made to provide the earliest and most accurate diagnosis possible along with appropriate therapy and close follow-up. This efficient design is an improvement, compared with a recent study that required 39 months to enroll 391 patients across 19 countries and 95 sites, with publication coming >5 years after enrollment began and >9 years after strategic planning [1]. Although this tour de force raises the bar for trial efficiency, enrollment still required 71 sites in 10 countries, and results are only being published now, 6 years after strategic planning. Because of the small aspergillosis drug market and escalating trial costs, such an expensive endeavor raises concerns that potentially life-saving antifungal therapies may not be developed. The study also suffers from the limitations of conventional RCTs for aspergillosis that rely on current disease definitions [7] and outcome criteria [1], including uncertainties regarding whether patients have active aspergillosis at enrollment and how best patient response can be assessed. By including historical or contemporary controls, salvage trials for patients with aspergillosis “refractory to” or “intolerant of” standard therapy are advanced as alternatives to RCTs. In addition to major biases resulting from such controls [8, 9], this strategy suffers from a fatal flaw: that of enrolling “refractory” patients with clinical/radiological worsening during neutrophil recovery [8, 10, 11]. When such patients improve, they are considered “responders” to the salvage agent, leading to regulatory approval. My colleagues and I [12] have shown that such patients may be responding before enrollment and that their transient deterioration represents immune reconstitution and inflammatory syndrome (IRIS) related to recovery of neutrophils. In aspergillosis, host trumps drugs and bugs.The primary determinant of outcome in aspergillosis is host immunity, as shown by experimental studies, by the unique susceptibility of immunosuppressed patients to aspergillosis, and by the strong relationship between outcome and resolution of immunosuppression, including remission of underlying disease [13]. Host confounds outcome assessment, unless anAspergillus-specific marker is repeatedly applied.Immune recovery may confound outcome assessment, as occurs in IRIS [12]. Continuing the same antifungal therapy in patients with IRIS was possible because of the normalization of daily serum Aspergillus galactomannan index (GMI) during clinical/radiological deterioration. Galactomannan is an Aspergillus-specific polysaccharide released during aspergillosis, and quantitative GMI values correlate with outcome [14, 15]. Thus, repeated use of the GMI allows, for the first time, determination of the time of response and of the contribution of immune reconstitution to outcome [12, 15]. Despite its remarkable efficiency, the study by Cornely et al. required dozens of enrolling sites in 2 continents and 6 years to complete because of a low incidence of disease, patient heterogeneity, and patient attrition. Aspergillosis is uncommon, the number of investigators is limited, and several treatment strategies are concurrently studied, all competing for the small pool of patients [16]. Minimizing uncertainties is central to the drug approval process. Unfortunately, conventional RCTs for aspergillosis are plagued with uncertainties. At enrollment.First, there is patient and disease heterogeneity. Patients with aspergillosis experience various diseases, and within groups (e.g., cancer), differences include disease and its status, immunity, cancer therapy, and others [16]. Such heterogeneity calls for a large study sample size and stratification for prognostic factors, including disease status and allogeneic stem cell transplantation [2]. Second is diagnostic heterogeneity. Definitions of aspergillosis [7] have been critical in removing variability in trial reporting but remain unvalidated. Enrollment in aspergillosis trials is based on the presence of a halo sign, crescent sign, or cavitation on a CT, even in the absence of microbiologic documentation of aspergillosis [2]. Absent of such documentation, these findings should not be acceptable for enrollment; they are nonspecific, nonstandardized, and unvalidated [8, 9], and crescent signs and cavitation are associated with immune reconstitution and resolving—not active—aspergillosis [17]. Progressive infiltrates have also been included as enrollment criteria [10,11], despite evidence that transient worsening (i.e., IRIS) may develop among responders [12]. Unless serial microbiologic testing, such as determination of the GMI, is performed before enrollment, trials may include heterogeneous groups of responders and patients with persistent aspergillosis. During therapy.Variable immune recovery adds to heterogeneity after enrollment, with some patients experiencing prolonged neutropenia and others quickly recovering [18]. Subject attrition, if unrelated to death, is a major challenge for conventional trials of aspergillosis, as illustrated by the 39% rate of attrition in this study (366 patients were randomized, and 201 were evaluable) [2]; this is slightly higher than the 32% rate reported by others [1, 19]. At enrollment.Approximately 30% of enrolled subjects are excluded because of an inability to confirm the diagnosis [1, 2, 19], partly because the definitions of aspergillosis used to enroll patients [1,2, 19] include—but do not mandate—GMI testing, despite evidence that ∼60% of potentially eligible subjects may be missed without such testing [20–22] and that the GMI test's excellent performance in neutropenic patients [14,15] will capture the overwhelming majority of these patients [20–22]. At outcome evaluation.Because of conflicting clinical, radiological, or mycological data and comorbidities, data for an additional 4%–11% of subjects are lost as a result of an “indeterminate outcome,” according to the conventionally used global response composite end point [1, 2]. However, this end point is expert derived and unvalidated, and its criteria are subjective (e.g., signs and symptoms and percentage resolution of “attributable” radiological findings) or rarely available (e.g., repeated culture and histopathologic testing). The following scenario, which regards 100 potentially eligible subjects, highlights the importance of repeated GMI testing: absent testing, ∼60 patients may be missed [20–22], and 15 may not be evaluable because of an inability to confirm the aspergillosis diagnosis or to determine the outcome [1, 2, 19], leaving only 25 evaluable subjects. Death, a frequent aspergillosis-related event, is commonly not included in outcome [1, 19, 23, 24] because of the difficulties in attributing causality even with an autopsy examination and because of declining rates of autopsy. Because death before response may be due to drug toxicity, death should be an outcome variable. With proper sample size and randomization and with objective and Aspergillus-specific enrollment and outcome criteria, non–fungal infection–related deaths should be balanced between study arms. Finally, outcome evaluation is usually performed 12 weeks after the start of therapy [1, 23]. Unfortunately, before this end point is reached, a large proportion of enrollees die [25, 26], frequently of progressive cancer and comorbidities. Because of low autopsy rates and the assignment of death as treatment failure, in absence of autopsy [2, 25], many late deaths may be incorrectly reported as indicative of treatment failure. Earlier evaluation reduces this uncertainty and is possible, as suggested by observations that outcome differences can be recognized within 6 weeks after the start of therapy [8, 27]. A 12-week end point to assess toxicity is recommended. The 2-week evaluation reduces uncertainties related to late deaths but may introduce others, such as the assessment of patients with worsening radiological findings as having experienced treatment failure [2], without considering whether IRIS may be responsible for these transient findings. A conceptual framework for strategies likely to expedite future approval of aspergillosis therapies, with a focus on pulmonary aspergillosis in neutropenic adults with hematological cancer, is presented in table 1. Strategies aimed at limiting heterogeneity, decreasing attrition, and enriching trial populations with subjects likely to respond and survive can decrease trial duration and costs. Conceptual framework for strategies to expedite approval of therapies against aspergillosis in neutropenic adults with hematologic malignancies. Meet the surrogate end point, courtesy of the US Food and Drug Administration (FDA).To enhance trial efficiency, the FDA [35] recommends use of surrogate markers—that is, tests that can substitute for clinical events as tools to increase diagnosis specificity and provide objective outcome measures [36]. However, successful implementation of surrogate markers (such as GMI assessment) in aspergillosis trials is predicated on demonstrating that GMI correlates with objective clinical outcomes. Is serum GMI a surrogate marker for aspergillosis in neutropenic adults?Traditional drug approval is based on large trials that demonstrate that a drug is safe and has a beneficial effect on clinical outcome or on already validated surrogate markers for which the drug is intended as treatment. Comparators also need to have the same effect on the surrogate marker. According to the FDA Deputy Commissioner for Operations, however, rigid criteria have made it all but impossible to “validate” surrogates, and “no surrogate has ever met them” [37, slide 52]. Under “accelerated approval provisions,” the FDA accepts “unvalidated surrogate markers” as primary end points for approval, provided that “treatment is for life-threatening conditions” and that “benefit cannot be shown in conventional clinical trials over a reasonable period of time or cost.” The surrogate must also be “reasonably likely to predict clinical benefit” [38]. Serum GMI has been approved by the FDA as a diagnostic marker for aspergillosis in traditional registration trials [28, 29, 39], and it fulfills all criteria as a validated surrogate end point for aspergillosis outcome (table 2). Therefore, GMI could be considered a validated surrogate end point under traditional approval guidelines. Because aspergillosis is life-threatening, the GMI could also serve as an “unvalidated surrogate marker” under “accelerated approval provisions.” Serum galactomannan index (GMI) is a validated surrogate end point for aspergillosis and is superior to conventional methods for assessing outcome. Critical to validation is the demonstration of strong correlations between surrogate and objective and important clinical outcome using accepted methods, such as κ correlation coefficient [69]. Concordance is excellent when the κ is ⩾0.75 and good when it is 0.4–0.75. Applying the κ correlation coefficient for GMI and survival to 228 published cases of aspergillosis [14, 30–32, 49–56, 65, 67, 70–83] and to 53 patients cared for at my institution yields excellent concordance (κ, 0.827 and 0.9, respectively). Additional evidence favors GMI over accepted outcome markers (table 2). A word of caution: only serum GMI qualifies as a surrogate marker for diagnosis and outcome evaluation of aspergillosis. GMI from other sites (e.g., bronchoalveolar lavage, CSF, and urine) does not fulfill surrogacy criteria for diagnosis or outcome evaluation. Efforts to validate serum GMI in solid organ transplantation and postengraftment allogenic stem cell transplantation and to clarify the role of GMI values at other sites are needed. Defining the outcome of aspergillosis.To minimize uncertainties in the drug-approval process, a proposed definition of aspergillosis outcome is offered that is based on Aspergillus-specific outcome measures (GMI), objective and quantitative measure of fungal load (GMI), and important and objective outcome (survival). Success is defined as survival with persistently negative GMI results, in the absence of new aspergillosis-associated lesions (e.g., skin lesions with microbiologic and/or histopathologic evidence of aspergillosis), or as death in which the presence of aspergillosis has not been proven by autopsy (with microbiologic plus histopathologic examination). Failure is defined as persistently positive GMI results and/or death, unless aspergillosis is not demonstrable at autopsy. Limitations of GMI.False-positive GMI results occur after receipt of treatment with galactomannan-containing antibiotics [39], an issue that can be resolved by avoidance of use of these agents. “False-positive” and “false-negative” GMI results have been reported under other conditions [39]; the former has been reported among survivors of aspergillosis for whom GMI performance was compared with methods that are rarely available (e.g., histopathologic examination and/or culture); that are unvalidated, nonspecific, and not repeatedly performed (e.g., CT); or that are transient (e.g., halo sign on CT). Proper evaluation of test performance requires comparison of the test with the gold standard for this condition (i.e., autopsy examination for aspergillosis). When GMI has undergone this comparison, very low rates of false-positive results (1.3%) and false-negative results (2.6%) occurred [14, 30, 32, 49, 50, 53, 67, 71, 73], implying that most “false-positive” results represent true antigenemia (which is similar to cytomegalovirus antigenemia without disease) or result from incomplete diagnostic evaluation to document disease. Using an optical density of ⩾0.5 to determine positivity, false-negative GMI results rarely occur, except when prophylaxis with mold-active agents has been administered—an uncertainty that can be resolved by avoiding such prophylaxis. Like other researchers [30, 84], I use GMI-guided aspergillosis therapy because of the low incidence of disease, cost, number of adverse events, and rate of drug interaction with mold-active agents [85], as well as because of the excellent GMI performance in neutropenic adults using an OD ⩾0.5 [39, 66]. Here is a not-so-paradoxical effect [62]: increasing GMI in the face of improving survival and decreasing tissue damage was reported after echinocandin therapy in cases of experimental aspergillosis. This “paradoxical effect,” however, neither presents consistently in experiments [40–46, 86] nor is seen in humans [57]. Cornely et al. [2] demonstrated, with some caveats, that increasing the dosage of liposomal amphotericin B to 10 mg/kg per day (compared with 3 mg/kg per day) does not improve its therapeutic index in neutropenic adults with respiratory tract aspergillosis. I caution, however, against extrapolating these findings to other infections (e.g., zygomycosis and CNS aspergillosis). Cornely et al. [2] also raised the bar in trial efficiency. Yet conventional RCTs for aspergillosis therapy continue to be burdened by huge costs, not the least of which is the enormous cost of time: drug patents are expiring, other drugs are advancing, and patients still die of aspergillosis. Furthermore, conventional RCTs are plagued with uncertainties, partly because expert-derived definitions place little emphasis on aspergillosis-specific and objective markers of diagnosis and outcome. We are entering a new era in the study of mold infection in which surrogate end points and novel trial strategies will allow us to investigate therapies quickly and efficiently. It is incumbent upon us to heed the FDA's call for more-efficient trials [36] and to break the mold of conventional trial design for mold-active agents. I thank Drs. John Bennet, Maria Cecilia Dignani, Paul Gubbins, Marisa Miceli, John Rex, and David Stevens for reviewing the manuscript and providing advice. I thank Drs. Mazhar Rahman and Monica Grazziutti for assistance in reviewing the literature and Weizhi Zhao for performing the κ correlation analysis. This manuscript would not have been possible without the editorial assistance of Paul Duguid. Potential conflicts of interest.E.J.A. has received grant support from Astellas, Curagen, Enzon, Nuvelo, Orthobiotech, and Pfizer; is a consultant for Astellas, Gilead Sciences, Merck, Pfizer, and Schering Plough; and is a member of the speaker's bureau for Astellas, Gilead Sciences, Merck, Pfizer, and Schering Plough.