Answers are grounded in this paper and cite their sources.
Sir, It was with interest that we read the invited commentary by Hammarberg et al. (2016) on how to use and evaluate qualitative research. However, we were surprised to see them expressing disappointment ‘to find that evidence from research using qualitative methods was not included in the formulation of the guidelines’, referring to the ESHRE guideline on routine psychosocial care in infertility and medically assisted reproduction (Gameiro et al., 2015). This reply intends to clarify the position of the Guideline Development Group (GDG) regarding the use of qualitative research in general and in the delineation of best practice recommendations. First, we would like to make it clear that the GDG did not a priori decide to exclude qualitative studies. The decision not to rely on individual qualitative studies for formulating the recommendations was made as a consequence of following the rigorous 12-step methodology for writing guidelines outlined in the ESHRE manual (Vermeulen et al., 2014), and supported by the ESHRE methodological expert (N.V.). In the following paragraphs, we will further clarify the employed method. We started with considering all research relevant to answer each of the research questions underlying the recommendations. This means that quantitative and qualitative studies were equally considered. All the evidence gathered to answer each research question was evaluated based on the principles of evidence-based practice (Sackett et al., 2000), which rely on the design of the studies and on evaluation of the studies with design-specific criteria. These principles define qualitative research, but also case–control studies and case reports, as lower quality evidence to formulate good practice recommendations when compared with randomized controlled trials (RCTs) and meta-analysis. Describing all the ranking criteria is beyond the scope of this letter but the interested reader is directed to the publications of the centre of evidence-based medicine (OCEBM Levels of Evidence Working Group, 2011). This means all studies were evaluated, regardless of being quantitative or qualitative. We used the Criteria for Evaluating Qualitative Studies developed by the Qualitative Research and Health Working Group to evaluate qualitative studies (Bromley et al., 2002). After all studies were evaluated, we followed the a priori decision to only consider level 2 or higher quality evidence for formulating the recommendations and this resulted in excluding qualitative studies. Second, it is important to clarify that qualitative evidence was indeed considered when summarized in systematic reviews or collected within cohort study designs. Such evidence proved particularly relevant when addressing patient preferences of care. Third, it is explicitly stated in the Guideline when qualitative evidence offered a different perspective to the recommendation. In these cases the discrepancy was noted and relevant studies cited (for example, see p. 43 of the Guideline). In most cases, however, we found no qualitative studies that contradicted the recommendations. Finally, we would like to make clear that qualitative research methods are not the best methods by which to generate practice recommendations. As Hammarberg stated, qualitative methods ‘are used to answer questions about experience, meaning and perspective, most often from the standpoint of the participant’. Qualitative research is based on a small number of participants, is not particularly concerned with the generalization of findings and does not attempt to achieve replicable outcomes. It naturally follows that qualitative research methods are not the best to inform practice recommendations, which describe actions that are true or work best for the majority of (a specific group of) patients, that health professionals can implement and whose effect can be measured. Even qualitative researchers acknowledge the limitations of qualitative methods to answer clinical and policy questions (Greenhalgh et al., 2016). This does not mean that the GDG does not appreciate qualitative studies. For example, a qualitative study may be adequate to understand why a specific practice recommendation is not having the expected impact. Indeed, all GDG members are familiar and experienced with qualitative research methods. We are aware of the value of qualitative methods to capture real-life experiences of health, illness and medical interventions and have no hesitation in using them in the context of our own research, when these prove to be the most adequate methods to address the research questions at stake (Emery et al., 2004; Dancet et al., 2012; Mesquita da Silva et al., 2016; Schick et al., 2016). We acknowledge the value of qualitative research methods to advance knowledge about multiple health issues not suitable to be researched with quantitative methods (Greenhalgh et al., 2016). In summary, we hope we have reassured our colleagues that we did not disregard qualitative evidence in the guideline but merely weighted the appropriateness of the different types of evidence available to support the recommendations. All relevant qualitative studies were included, evaluated and used when it provided a perspective that was not evident from just using the quantitative data. S.G., E.D., M.E., U.V.d.B., and N.
No takes yet. Share an insight, caveat, or question.
Gameiro et al. (2016) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: