Clinical evidence for orphan medicinal products-a cause for concern?

BACKGROUND: The difficulties associated with organising clinical studies for orphan medicinal products (OMPs) are plentiful. Recent debate on the long-term effectiveness of some OMPs, led us to question whether the initial standards for clinical evidence for OMPs, set by the European Medicines Agency (EMA) at the time of marketing authorization, are too low. Therefore, the aim of this study was to quantitatively evaluate the characteristics and quality of clinical evidence that is presented for OMPs to obtain marketing authorization in Europe, using the new and validated COMPASS tool. METHODS: We quantitatively assessed the characteristics and quality of clinical evidence of the pivotal studies of 64 OMPs as described in the European Public Assessment Report and/or the Scientific Discussion document prepared by the Committee for Human Medicinal Products of the EMA. RESULTS: The 64 OMPs were altogether authorized for 78 orphan indications, for which 117 studies were identified as 'pivotal' or 'main' studies. In approximately two thirds of the studies, the allocation was randomized (64.8%) and a control arm was used (68.5%). Half of the studies applied some type of blinding. Only a minority (26.9%) of the studies included a Quality-of-Life (QoL) related endpoint, of which a third claim an improvement in QoL. Upon analyzing the quality of reporting, we found that some aspects (i.e. the endpoints, the sampling criteria, and the interventions) are well described, whereas other items (i.e. a description of the patients and of potential biases) are not reported for all studies. CONCLUSIONS: In conclusion, the pivotal studies that are the basis for marketing authorization of OMPs are a cause for concern, as they exhibit methodological flaws i.e. the lack of QoL-related endpoints as outcome, lack of blinding in the study design and the use of surrogate endpoints. Additionally, there are shortcomings in the reporting of those studies that complicate the interpretation. A more demanding regulatory process for OMPs is needed to guide evidence-based clinical decision-making.