Failures, Mistakes and Shortcomings of My Stroke Trials

Treatment and prevention of stroke depend on the results of well-planned and executed randomized trials. Trial methodology has developed over the last 30 years, and at present we have very sophisticated ways to perform randomized trials. I started to participate in my first randomized trial in 1989 after finishing my residency in neurology. Until 2013 I had participated in >100 trials in the acute treatment and prevention of stroke. Over the years I became aware of many shortcomings in the design, execution and interpretation of clinical trials in which I participated.In this paper I would like to give a few examples and describe how modern trial design helps to avoid these mistakes in the future. Quoting Winston Churchill: ‘All men make mistakes, but only wise men learn from their mistakes' (table 1).In 1989 carotid surgery for symptomatic high-degree stenosis of the internal carotid artery was well established, and 2 randomized trials (NASCET and ECST) were ongoing [1,2] comparing carotid endarterectomy and medical treatment. The role of carotid surgery in asymptomatic carotid stenosis was unclear. The prospective study by Hennerici et al. [3] indicated that the stroke risk was relatively small. A group of vascular surgeons, neurologists and clinical trial experts therefore applied for a grant from the German Ministry of Science and Technology to perform a randomized trial comparing carotid surgery with best medical treatment in patients with asymptomatic carotid stenosis. After the grant had been approved, we called a meeting with the surgical centers in Germany from whom we knew that they had enough expertise in carotid surgery. Unfortunately the participating vascular surgeons thought that withholding surgery from patients with severe carotid stenosis and patients with bilateral disease would be unethical. Therefore the trial only randomized patients with unilateral moderate carotid stenosis. In addition the trial was underpowered and predictably failed to show a difference between surgical and medical treatment [4].The lessons to be learned are as follows:(1) A good study acronym (CASANOVA) does not replace a proper study design(2) Sometimes it is better not to start a trial than to accept compromises which decrease the likelihood of achieving positive resultsCerebral ischemia triggers excessive presynaptic release of the excitatory neurotransmitter glutamate and the energy uncoupling prevents active removal of glutamate from the synaptic cleft. The excess of glutamate overstimulates postsynaptic receptors such as the N-methyl-D-aspartate (NMDA) receptor, thereby opening the associated ion channel and allowing sodium and calcium ions to enter the cell whilst potassium flows outwards. The calcium influx activates intracellular proteases and initiates a cascade of further energy consuming and ultimately lethal processes. Drugs were developed for most of the assumed mechanisms in cerebral focal ischemia and tested successfully in animal experiments of transient or permanent cerebral ischemia. Initially only experiments in healthy young rats were performed. The Stroke Therapy Academic Industry Roundtable issued a number of recommendations how to improve animal experiments including a randomized blinded design, use of placebo and experiments in older rats with vascular diseases or even primates [5,6,7,8,9,10]. Despite more than 1,000 papers reporting positive results in animal experiments, until today none of the >150 randomized trials in human stroke investigating neuroprotective therapy has been successful.Selfotel is a selective NMDA antagonist. Selfotel was effective in animal models of stroke, when young rats without concomitant vascular risk factors were examined. Dose-finding studies did not reveal major problems in terms of adverse events [11]. The phase III trial planned to recruit 1,000 patients with acute ischemic stroke. The trial was terminated prematurely by the data safety monitoring board when 3-month data from 476 patients were available [12]. Mortality at this point in time was 13% with selfotel and 5% with placebo.The lessons learned from ASSIST were as follows:(1) The study in humans was started with very little data from animal experiments(2) The results in animals were not gained in placebo-controlled trials and the data were not replicated by an independent laboratory(3) The study started with not enough safety data in elderly patients with vascular risk factors or vascular diseases(4) Serious adverse events like death were not reported immediately, which caused a delay in terminating the study. Modern GCP guidelines require communication of serious adverse events within 24 hLubeluzole is a benzothiazole derivative that has been shown experimentally to preserve neurological function and reduce infarct volume in animal models of focal brain ischemia [13,14]. The neuroprotective activity of lubeluzole is related to its ability to alter the biochemical cascade that leads to irreversible neural damage in the penumbra, by preventing an increase in extracellular glutamate and normalizing neuronal excitability in the peri-infarct region [15]. Experiments with cultured embryonic hippocampal neurons further indicated that the drug inhibits glutamate-induced nitric oxide-related neurotoxicity [16].Lubeluzole was investigated in 2 phase III trials [17,18]. The dose was smaller than anticipated from the results of animal experiments because higher plasma levels in humans resulted in QT time prolongation. The time window for inclusion was 8 h, and the primary end point was the score of the Barthel index at 12 weeks. The core analysis group was defined by a time window of 6 h and age <75 years. The trials were negative for the primary and secondary end points.Lessons learned from the lubeluzole program:(1) Lubeluzole was the first neuroprotective substance investigated in a proper phase II program(2) Due to anticipated QT time prolongations with higher doses of lubeluzole in humans, the drug was probably underdosed. A post hoc analysis after the trial program had been terminated showed a significant relationship between plasma levels and efficacy [unpubl. data](3) The trial also showed that patients with very severe strokes and elderly patients have a poor prognosis independent of treatment assignment [19](4) The Barthel index is a poor outcome parameter. The index has a U-shaped distribution and therefore a poor discrimination threshold [20]Gavestinel (GV150526) is a potent and selective antagonist at the glycine site of the NMDA receptor. It reduces infarct size in rats by slightly more than 50% when administered intravenously within 6 h after the onset of middle cerebral artery occlusion [21]. The compound was well tolerated by volunteers and by stroke patients [22]. Because gavestinel does not affect platelet or coagulation function, and has beneficial effects in an animal model of traumatic intracranial hemorrhage [unpubl. data], the drug has been thought suitable for widespread use without prior neuroimaging. Phase II studies with gavestinel were unsuited to assess outcome trends because there was imbalance in prognostic variables between groups, the time window to treatment of 12 h was rather long, and the follow-up period of 1 month was too short [23]. However, the drug was very well tolerated. Two phase III placebo-controlled trials with gavestinel were performed in the USA (n = 1,367) and the rest of the world (n = 1,804) [24,25]. The inclusion time was 6 h and patients were treated for 3 days. Patients with ischemic as well as hemorrhagic strokes were included. Patients were stratified according to age and stroke severity. Limb weakness in the National Institutes of Health Stroke Scale was required. Both studies failed to show a difference between gavestinel and placebo.Lesions to be learned from the gavestinel program:(1) Patients with ischemia and cerebral hemorrhage should not be randomized in the same trial(2) The absence of central side effects could indicate that a neuroprotective substance does not cross the blood-brain barrierNXY-059 was a free-radical-trapping agent, which had undergone the most extensive preclinical development in experimental stroke. The substance was effective in young and old rats as well as in primates and showed efficacy in animal experiments both for transient and permanent middle cerebral artery occlusion [26,27]. The phase III program had a 6-hour inclusion time window and allowed patients receiving thrombolysis with recombinant tissue plasminogen activator to be randomized. The primary end point was the distribution of the modified Rankin scale. The first trial randomized 1,722 patients and was positive for the primary end point but failed to achieve superiority versus placebo for a number of secondary end points [28]. The second trial with 3,306 patients and a similar study design was neutral as was the pooled analysis of the 2 phase III studies [29]. The drug was well tolerated and showed no difference in adverse events between active drug and placebo.Lessons to be learned from SAINT:(1) One positive placebo-controlled trial does not prove that a neuroprotective drug is effective in the treatment of ischemic stroke(2) Absence of side effects of a neuroprotective drug might indicate that the drug does not reach the target areaAbciximab is a platelet glycoprotein IIb/IIIa receptor antagonist and was effective in patients undergoing percutaneous coronary interventions [30] and those with unstable angina [31]. The hypothesis was that this drug would be at least as effective as recombinant tissue plasminogen activator in ischemic stroke and should have a longer time window. We conducted a phase II dose-finding trial in 54 patients on the active drug and 20 on placebo [32]. The drug seemed to be safe and showed a trend towards an improved outcome.An international, randomized, placebo-controlled, double-blind phase III trial tested the intravenous administration of abciximab in acute ischemic stroke with stratification for time since onset and stroke severity [33]. The planned enrolment was 1,800 patients. The inclusion time was 6 h. The primary efficacy measure was the dichotomous modified Rankin scale score at 3 months as adjusted to the baseline severity of stroke among subjects in the primary cohort. The primary safety outcome was the rate of symptomatic or fatal intracranial hemorrhage that occurred within 5 days of stroke. The trial was terminated prematurely after 808 patients in all cohorts had been enrolled by recommendation of an independent safety and efficacy monitoring board due to an unfavorable benefit-risk profile with an increase in cerebral bleeding complications. At 3 months, 33% of patients assigned to placebo (72/218) and 32% of patients assigned to abciximab (71/221; p = 0.944) in the primary cohort were judged to have a favorable response to treatment.Lessons to be learned from the AbEST trials:(1) Small phase II trials with antithrombotic therapy in acute ischemic stroke will not be predictive of safety in a larger trial(2) More potent antithrombotic therapy will lead to more intracranial bleeds and offset any potential beneficial effectPatients who suffered a transient ischemic attack or ischemic stroke have a high risk of recurrence. In patients with platelet-rich thrombi originating from atherosclerotic plaques, secondary prevention is performed by antiplatelet therapy. Patients with a cardiac source of embolism are anticoagulated. Most placebo-controlled trials until 1990 used aspirin. The majority of the trials missed the primary end point [34], and therefore aspirin was not approved for the secondary prevention of stroke.The combination of aspirin and dipyridamole was superior to placebo in secondary stroke prevention in the first European Stroke Prevention trial [35]. The Second European Stroke Prevention (ESPS-2) study included 6,602 patients to receive aspirin monotherapy (25 mg b.i.d.), extended-release dipyridamole (200 mg b.i.d.), the combination of aspirin and extended-release dipyridamole, or placebo for a mean follow-up of 2 years [36]. Aspirin/dipyridamole significantly reduced the primary end point stroke compared to aspirin monotherapy (relative risk reduction, RRR, 23%, 95% confidence interval, CI, 0.65-0.97) and to placebo (RRR 37%, 95% CI 0.48-0.73). Major bleeding complications were seen more frequently with aspirin and the combination aspirin and dipyridamole, whereas dipyridamole monotherapy had a similar bleeding rate compared with placebo. Cardiac events occurred in a similar frequency in the groups treated with dipyridamole compared to that treated with aspirin [37]. This was the first large-scale trial which showed that low-dose aspirin is effective in secondary stroke prevention leading to the approval of aspirin for this indication. The original paper from ESPS-2 was submitted to The Lancet but was rejected. One reviewer claimed that it was unethical to use placebo in such a trial, despite the fact that aspirin had not shown efficacy in earlier trials. Another point of critique was the choice of the primary end point which was stroke and not the combined end point of nonfatal stroke, nonfatal myocardial infarction and vascular death. Our power calculation when we started the trial indicated based on earlier trials that antiplatelet therapy would not decrease mortality, and therefore including vascular death would dilute the trial.Dipyridamole can induce headache in the first days of intake. This adverse event resulted in a considerable number of early dropouts in ESPS-2. Unfortunately we had no strategy in place how to handle this problem. Later we learned that a slow titration of dipyridamole can avoid this problem [38]. Another unpleasant surprise was the fact that our monitors detected one center in the Netherlands in which 446 patients had been falsified. ESPS-2 was one of the first randomized trials with systematic monitoring. Later it turned out that the same investigator had falsified data in 12 earlier trials on stroke and headache. ESPS-2 was heavily attacked because of the faked patients. Our team, however, was the first which identified our criminal colleague. As a consequence he lost his medical license.Lessons learned from ESPS-2:(1) Object if a good manuscript is rejected by a major journal(2) Anticipate adverse events and implement a strategy to deal with them(3) Monitoring is of the utmost importance to avoid scientific misconductClopidogrel is a thienopyridine derivate and was investigated for the prevention of stroke, myocardial infarction and vascular death in the CAPRIE trial [39]. CAPRIE randomized 19,185 patients with recent ischemic stroke, myocardial infarction or symptomatic peripheral arterial disease to clopidogrel 75 mg or aspirin 325 mg. The trial showed a modest RRR of 8.7% (95% CI 0.3-16.5) in favor of clopidogrel. In subgroup analyses only patients with peripheral arterial disease showed a clear benefit for clopidogrel over aspirin. This created a fierce scientific debate. Many reimbursement bodies in Europe refused to pay for clopidogrel for secondary stroke prevention [40]. Scientists argued that the overall result of the trial should be considered since subgroup analyses were introduced later.Lessons learned from CAPRIE:(1) It is dangerous to mix 3 different patient populations into one trial(2) Too many post hoc analyses driven by marketing interests might compromise the study(3) The poor number needed to treat was never properly communicatedIn cardiology the combination of aspirin plus clopidogrel was superior to monotherapy to prevent vascular events in patients with acute coronary syndromes [41] or undergoing percutaneous coronary interventions [42]. The combination, however, had a higher rate of major bleeding complications. We embarked on the MATCH trial in which we compared clopidogrel monotherapy with the combination of aspirin plus clopidogrel in 7,599 high-risk patients with recent ischemic stroke or transient ischemic attack. The trial showed a 6.4% RRR for the combined vascular end point and hospitalization which was statistically not significant [43]. We observed a significant increase in life-threatening and major bleeds with combination therapy. MATCH included 52% of patients with small vessel disease. Whether this population benefits from antiplatelet therapy at all is not clear. In a study which included only patients with lacunar strokes, clopidogrel plus aspirin was not superior to aspirin monotherapy and again showed a higher bleeding risk [44].Lessons learned from MATCH:(1) A trial with a mean follow-up of 18 months is too short(2) An enriched high-risk population might lead to bias(3) MATCH included a high percentage of patients with diabetes. Patients with diabetes might respond differently to antiplatelet therapy compared to patients with normal glucose metabolism(4) Patients with small vessel disease might not benefit from antiplatelet therapy(5) The study should have compared combination therapy with aspirin and not clopidogrel(6) More potent antiplatelet therapy will lead to an increase in bleeding complicationsClopidogrel was marginally superior to aspirin in the CAPRIE trial [39] and aspirin plus dipyridamole was clearly superior to aspirin in the ESPS-2 [45] and ESPRIT trials [46] for secondary stroke prevention. A number of meta-analyses, in particular one network meta-analysis, suggested that the combination of aspirin and dipyridamole should be superior to clopidogrel [47]. In this meta-analysis the combination of aspirin and dipyridamole was more effective than thienopyridines (odds ratio 0.84; 95% CI 0.73-0.97). The PRoFESS trial randomized 20,332 patients after ischemic stroke to either clopidogrel 75 mg once daily or aspirin 25 mg plus slow-release dipyridamole 200 mg twice daily. After 2.15 years of follow-up, the primary end point, recurrent stroke, was the same in both treatment groups. The risk of major bleeds was higher with combination therapy.Lessons learned from PRoFESS:(1) We should not base our treatment decisions on indirect comparisons with results from randomized trials(2) In most studies in cardiology and neurology, combined antiplatelet agents result in a higher rate of major bleeds than monotherapy [48]Most patients who are admitted with an acute stroke have increased blood pressure. High blood pressure has a negative impact on outcome [49]. Therefore we performed a phase II safety study and planned to randomize 500 patients with acute stroke and blood pressure >180 mm Hg to either candesartan or placebo. The trial was terminated prematurely when 342 patients had been randomized due to a significant imbalance in end points. When the study was unblinded, it turned out that, contrary to our concerns, antihypertensive treatment was superior to placebo [50]. This resulted in a large phase III trial. SCAST recruited 2,029 patients with acute stroke and blood pressure >140 mm Hg. Patients were either treated with candesartan or placebo for 7 days. The trial was unable to find a benefit of blood pressure lowering with candesartan in acute stroke and raised blood pressure [51].Trial design and conduct have made tremendous progress in the last 30 years. Learning from the mistakes and failures in the past will further improve ways we design and perform treatment trials in acute stroke and stroke prevention. A shown in table 1, mistakes are not always a bad thing.I would like to thank many colleagues and friends who planned and performed clinical trials with me: H.P. Adams; G.W. Albers, P. Amarenco, P. Bath, J. Bogousslavsky, N. Bornstein, M.G. Bousser, L.M. Brass, T. Brott, B.P.L. Chan, L. Cuhna, L. Csiba, A. Davalos, S.M. Davis, G.A. Donnan, J.D. Easton, C. Estol, M. Fisher, G.A. Ford, P. Gorelick, J. Grotta, W. Hacke, G. Hankey, M. Hennerici, M. Kaste, K.R. Lees, D. Leys, P. J. A. J. D. J. C. which was my that well was their I also to thank the we had in the for in clinical to or and for was by and The of at the from the German German Ministry of and European and has no and does not of any the past as of Stroke and the Treatment of the German as of and on the board of Lancet European and