Increasing value and reducing research waste in obstetrics: towards woman‐centered research

After being awarded Archie Cochrane's infamous ‘wooden spoon’ for the limited application of randomized controlled trial evidence to questions of clinical practice, obstetricians were inspired to action and ultimately became leaders in the development of evidence-based medicine1. Although the importance of randomized controlled trials and the systematic synthesis of their results to clinical practice is now undisputed, concern has moved to the quality of the design and conduct of primary trials and observational studies, completeness of reporting and the relevance of the research to clinical practice. It has been estimated that around 85% of all medical research funding is actually wasted, a staggering figure thought to be over US $80 billion2, but the impact of research waste goes beyond misuse of finite financial resources. Incomplete reporting, duplicated work, poorly conducted trials and delayed synthesis of existing evidence have led to substantial barriers in the identification of both beneficial new therapies and severe adverse effects associated with existing treatments, leading directly to patient harm3. As we seek to further integrate research within clinical settings, we must face the ethical implications of unaddressed research waste and commit to improving the quality of research in the same way that we strive to improve the quality of direct clinical care. In a landmark series published in The Lancet in 2014, 17 recommendations were proposed for the reduction of research waste across five stages of research production, dissemination and implementation4-8. It is not our intention to revisit the comprehensive assessment and rationale of these recommendations, rather to consider how in the intervening years the field of obstetrics has responded to the challenges, to identify those issues peculiar to our specialty in seeking to improve the process of research production, dissemination and implementation, and to explore avenues for future development. Women in general are under-represented in clinical trials, and pregnant women especially so9. This is despite the fact that pregnant women still develop health conditions that require treatment, and most medications administered in pregnancy, both over the counter and by prescription, are off-label and unlicensed for use in pregnancy10. The clinical use of drugs untested in pregnant women shifts the risk from monitored small populations in research contexts to uncounted and highly variable use in clinical practice. It is exactly this lacuna that contributed to the scale of the thalidomide disaster, and yet pregnant women are still rarely included in drug trials. This issue was recently brought to the fore again when, during the Ebola crisis, pharmaceutical companies, governments and humanitarian organizations collaborated with remarkable speed to bring Ebola vaccines into clinical trials – and yet, despite the fact that pregnant women were particularly vulnerable to Ebola and that perinatal mortality in Ebola has been near total, pregnant women were initially excluded from the trials of the vaccine11. This situation was remedied eventually, and forcibly illustrates the case made in the ethical guidelines of the Council for International Organizations of Medical Sciences (CIOMS), which state that ‘research designed to obtain knowledge relevant to the health needs of the pregnant woman must be promoted’. The STRIDER trials aimed to evaluate sildenafil as a treatment for fetal growth restriction, a treatment that is already creeping into common practice with the backing of observational data and physiological plausibility. Stopping early the trial in The Netherlands, when evidence emerged that sildenafil was actually associated with increased fetal risks, has changed clinical practice and demonstrates clearly how the responsible conduct of randomized controlled trials can protect women and children around the world12, 13. The blanket exclusion of women from studies that offer them, and their future families, important clinical benefits arises from a paternalistic determination that women are unable to assess risk for themselves, and fails to take into account the fact that acceptable risk is contextual and individual, as in the Ebola vaccine trials. For obstetricians, there is an ethical imperative to advocate for the inclusion of pregnant women in clinical research and lead the way in developing responsible ethical frameworks to allow women to make their own informed choices and access the benefits of research participation. Fortunately, most mothers experience uncomplicated pregnancies and even relatively common obstetric disorders are rare in comparison with conditions such as diabetes, hypertension and chronic renal disease within the general population. Fetal growth restriction secondary to placental insufficiency may be one of the most common conditions presenting to fetal medicine specialists, but on a population level it is rare enough to have been awarded orphan drug status by the European Medicines Agency (EMA) (which is given only when a disease is classified as ‘rare’, affecting fewer than 5 in 10 000 Europeans)14. In a field in which targeted conditions may be rare, it is of paramount importance to ensure that all research performed is usable. In obstetrics, we deal with a human imperative that is sometimes stronger than the will to survive: the will to reproduce and to protect our offspring. Patient priorities in the field of obstetrics, in which the interests of the fetus and those of the mother are not always in harmony, are often radically different compared with those in other disciplines15. Personal and cultural perspectives play a strong role in the interpretation and choices of women and their families in challenging clinical scenarios. How can an obstetrician determine the relative importance of the risk of a procedural complication for the mother against the risk of harm to the fetus from non-intervention? Might the lived experience of making those decisions not provide a more valuable form of evidence than the clinicians' perspective of observing patients go through these experiences16? In high-risk obstetrics, in which challenging and value-laden decisions about care are frequently required, we are obliged to provide the best possible evidence in the fullest sense of the word, addressing all the concerns of the patients; research work that does not meet this purpose is wasted indeed. A traditional paradigm for shared decision-making involves integrating the best available evidence with the personal and cultural perspectives of the patient in order to make decisions about their care. This model assumes inherently that the evidence itself is unaffected by cultural and personal biases, even those of the researchers and clinicians who have chosen what to research and how. When we acknowledge that this cannot be the case, we must also accept that high-quality evidence can be achieved only when it is produced with and for the people who will be using it, rather than generated independently by researchers to be presented to patients. Consciously opening the process of choosing research directions to as diverse a field of stakeholders as possible generates ideas that would not always have been identified as priorities by researchers17. For this reason, the James Lind Alliance formally brings together patients and healthcare professionals to prioritize the most pressing research uncertainties. The results of priority-setting exercises for infertility, preterm birth, stillbirth and multiple pregnancy generated questions covering not only new treatments and technologies, but also topics ranging from empowering women to discuss their concerns with care providers to the holistic care of families after experiencing adverse outcomes18-20. Additional priority-setting exercises are underway across obstetrics and gynecology, including hypertension in pregnancy. The aim of these partnerships is not to restrict researchers from pursuing genuinely novel treatments and technologies, nor to generate questions that could each be addressed in discrete trials, but to formalize the core principle that research should serve our patients. By establishing as a norm the need to be collaborative in agreeing priority research questions, our specialty can ensure that the use and applicability of novel therapies and technologies are considered from the very earliest stages of investigation and development. Negative or non-productive findings shared openly are not research waste, but unreported, unnecessarily duplicated, poorly analyzed or inaccessible research is. Changing the systemic incentives in terms of funding, publication and career development that prioritize the publication of statistically significant work over important null findings is key to reducing research waste. We should not fear to investigate questions to which the answer may be ‘no’, but we should avoid researching questions to which the answer would not matter either way. The involvement of patients and stakeholders should not stop after identifying important research questions; their involvement should also be central in study design, patient recruitment and participant retention15. Involving people with personal experience of the condition investigated has been shown objectively to improve trial recruitment and retention, but is also likely to improve the experience of trial participation and utility of the research output in ways more difficult to quantify21. We know that the patient agenda is not fully explored in most clinical encounters, and yet, it has a major impact on patients' choice to continue or modify treatment. Research designs that incorporate qualitative exploration of patient experience and priorities are more likely to generate information that is useful and applicable in the real world of multiple concerns and priorities16. Changing the research design paradigm to involve patients and other stakeholders is not only beneficial for improving the quality of research and reducing research waste, but also ethically imperative on the grounds of justice, autonomy and beneficence. There is a great wealth of patient data as yet unused or with unrealized potential from unpublished or incompletely reported randomized controlled trials, and this is the subject of the All Trials campaign22. Around half of all trials are estimated to remain unreported, with academic sponsors performing particularly poorly23. Considering this disappointing level of performance, it is legitimate to wonder if it is ethical to approach women to participate in research at all. Prospective registration of trials can ensure more complete reporting of data, particularly adverse events, than is achieved in journal publication alone, and it also allows sponsors and researchers to be held accountable for trials started but not reported24. Prospective registration and protocol sharing for trials is welcome, but when over 90% of published research is observational25, to address trial registration and reporting alone is to ignore the majority of our evidence base. This is particularly true in obstetrics, since the historical exclusion of pregnant women from controlled trials leads to an even greater dependence on observational data. Prospective registration of observational studies is recommended in the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guidelines26 and is feasible in a number of online registries; however, this is not yet common practice. It may not be reasonable to expect that all observational studies undertake formal registration. There are sound concerns that the heterogeneity of methods and purpose of observational research makes it impractical to apply to all observational studies a single standard for prospective registration, and this could actually limit exploration of new ideas27. Major journals, however, have made it clear that they expect large, hypothesis-driven observational studies to provide protocols and prospective registration details in the same way as randomized trials do28. If the primary evidence base benefits from prospective registration, prospective registration of systematic reviews is also important to reduce selective reporting in evidence syntheses. The PROSPERO registry of systematic reviews can provide transparency in the process and accountability for the study, and prevent unintended duplication of reviews29. Unfortunately, as demonstrated by the recent publication of five systematic reviews on the use of chewing gum after Cesarean section30-34, all showing a benefit and none yet having a noticeable impact on clinical practice, the availability of a platform for prospective registration of review protocols does not yet preclude research waste in this area. The impact of any tool is dependent on its uptake by researchers and, clearly, use of the PROSPERO registry is not yet fully established. Perhaps, in addition to requiring that all systematic reviews demonstrate a prospectively agreed protocol in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, authors should also be asked to demonstrate a search prior to commencing their review for similar ongoing projects and provide evidence for the novelty or relevance of their work35. Even before systematic reviews, researchers developing new trials have an obligation to consider not only what research priorities exist, but also what is currently known and how their work might advance that knowledge. Given that informal literature reviews are frequently found to be selective and avoid contradictory studies, the gold standard for this process should be performing a formal systematic review prior to embarking on a new trial3. Of three new trials of therapeutic interventions in obstetrics published in this journal last year, all made reference to prior work in their field but only one referred to a systematic review of the literature on the topic36-38. Even in this study, the systematic review mentioned was published after the trial recruitment had ended, so clearly it was not used to inform the trial design. Carrying out a systematic review during the development of a trial would help inform the study design and avoid research waste. Our specialty is drowning in numerous studies of little to no clinical significance and the difficulty in comprehensively assessing the breadth of the evidence on any particular question is substantial39. No matter how well it is registered, reported and conducted, research is of use only if it is accessible to the end user and implemented into clinical practice40. The gold standard of evidence synthesis is conducting systematic reviews and meta-analyses, but limitations relating to the primary studies (both trials and observational reports), the statistical methods of summarizing findings and the accessibility of review findings to clinicians and the public can limit their impact on clinical practice41, 42. A number of new approaches are being developed to address the limitations of traditional meta-analysis, including the development of core outcome sets (COS), use of individual participant data (IPD), umbrella reviews, and prospective and network meta-analyses. A particular challenge in evidence synthesis is heterogeneity between studies in case definitions and outcome reporting43-46. Consensus definitions ensure that the populations in each trial are comparable47-49 and minimum reporting sets can ensure that key population information is described in primary reports50. Defining clearly and consistently the population investigated is key to determining the clinical applicability of trial results and summarizing results across studies. Recent attention to consensus on the selection, collection and reporting of outcomes in randomized controlled trials has set the stage for new work that uses COS in order to increase transparency, accessibility and ease of evidence synthesis. The Core Outcomes in Women's and Newborn Health (CROWN) initiative is a collaboration of over 80 specialty journals, including Ultrasound in Obstetrics and Gynecology, which have established the importance of COS and have committed to supporting their development, dissemination and implementation39, 51. The Core Outcome Measures in Effectiveness Trials (COMET) initiative allows prospective registration of COS, and 98 projects have been registered within the obstetrics and gynecology field, just under 10% of the entire registry. COS have been developed for a range of conditions relevant to obstetrics, including pre-eclampsia, preterm birth and twin-to-twin transfusion syndrome52-57. In the development of COS, healthcare professionals, researchers, patients and other stakeholders come together to agree on the outcomes that will be most useful. It is important that COS are developed using robust consensus science methods to ensure they reflect the perspectives of all key stakeholders58. Embedding this perspective should prevent the practice of P-hacking, which is the selective reporting of outcomes or statistical and reporting of issues have been identified across a range of conditions relevant to obstetrics and Reporting of data in trial not only accountability and complete reporting, but also allows researchers to avoid the limitations of study case definitions and outcome reporting in primary study by the data for evidence The use of in evidence synthesis allows increase of the investigation of the of individual patient with treatment and of outcomes not included in the primary study has been described as the for evidence synthesis it use of all data and of across studies, but the can be and In addition to and for meta-analyses, in practice, the most significant for performing such studies is the difficulty in primary researchers responsible data sharing but the to for data is rather In one on of pre-eclampsia, studies were identified through a systematic literature but only were ultimately made available for with only of authors and of those ultimately not sharing their data or the data being found to be In in data data Research funding and are leading a with to supporting trial registration and data since The has only published new trials in which a to reasonable is and the International of Medical has formally the that sharing individual data is ethically and they are to the in their in of more collaborative In the field of obstetrics, in which the benefits of shared data are so significant and the ethical imperative so we should be leaders in the development of for collaboration that both the of unused data and the work of the researchers who designed and the data. A useful is the International of together more than researchers from who have contributed data from over pregnancies to be investigated using and on this the challenge is for individual researchers to the of data of the initiative has been the publication of a comprehensive review of all the published systematic reviews on for pre-eclampsia, which the ongoing of small studies with case definitions and further development in this This form of a of the evidence and, if performed should incorporate an assessment of the of in findings of individual reviews that can inform the decision-making of healthcare providers and their patients. A network allows as well as direct comparison of existing treatments, to a range of Recent a review of use for and a network of Trials and observational studies in both of these but they rarely more than or three a of the field and comparison of interventions than a traditional would but relatively statistical A key of is that only published trials are available for inclusion and the results of the trials are known prior to the of evidence synthesis. A prospective to address this issue by of prospective trials to search for and identify trials prior to their in order to prospectively the meta-analysis, including any The need for research data does not apply only to however, given that data is of quality in trials than in observational studies, having access to high-quality trial data is the to The should be to stop valuable data on outcomes of rare conditions in pregnancy by pregnancy data in or small studies reporting outcomes with of When rare conditions are single rarely have enough to identify outcomes or the of treatment choices with patient and the on clinical or for rare for the and registry for or those designed for rare could increase the number of available for prospective registration and such offer the potential for treatment rare from treatment and valuable avenues for future are of the work that is already being to reduce research waste in our We can to including the initiative and numerous priority-setting projects leading the way in collaborative work, and ultimately improving the information and care we can provide to mothers and there is work to and systemic barriers to to the of for the individual participant data. We need to consider a cultural across the from and researchers to the that poorly conducted, incompletely reported and ultimately wasted transparency, from the and design of studies through to the review of is likely to be a key of this cultural Prospective registration of trials and systematic reviews, a collaborative approach to data methods of evidence and of complete and accessible study reporting are all important The we all need to however, is mothers and at the of process in the design, production, dissemination and of We have to the of care in clinical research should be the in developing the evidence base to this care.