Response to ‘Comment on ‘Measuring the impact of medicines regulatory interventions – systematic review and methodological considerations’ by Goedecke et al.’

We would like to thank Christine Lu and Stephen Soumerai for their comments 1 on our systematic review of studies measuring the impact of medicines regulatory interventions 2. The primary aim of our systematic review was to highlight important methodological challenges and to inform the conduct and reporting of future studies evaluating the impact of regulatory interventions. Although different study designs and analytical methods are described, our systematic review does not assess their methodological quality and no selection criterion was based on quality. We agree that study designs with single observations before and after an intervention without any controls and simple cross-sectional designs are weaker study designs and typically provide lower quality evidence compared with other methods such as interrupted time series (ITS) and Joinpoint regression approaches. However, our systematic review intended to highlight that despite the importance of pharmacovigilance interventions on public health, relatively few studies have attempted to examine their impact. Of those which have, a significant proportion have been conducted using weaker study designs that ignore preintervention and postintervention trends and should not be routinely recommended first line (only 42.5% of studies identified used a statistical regression based approach) 2. It would not have been possible to highlight this important point if such studies had been excluded from our systematic review. As highlighted in the comment, the study by Lu et al. 3 reporting ITS analysis of changes in antidepressant use and suicidal behaviour after FDA warnings and media coverage was not included in our systematic review. We agree that this study assessing the impact of FDA regulatory interventions is highly relevant and should be included. Despite identifying 27 studies examining the impact of regulatory interventions with antidepressants, the study by Lu et al. was not identified by our electronic search strategy. This highlights another important point raised by our systematic review that there is a lack of common definitions and consistent terminology to describe and identify such impact studies, together with the variability in the reporting of analytical methods. For example, the study by Lu et al. was indexed using the specific MeSH term 'Antidepressive Agents/Adverse Effects' while our initial query in MEDLINE and EMBASE used general pharmacovigilance MeSH terms ('Pharmacovigilance' OR 'Drug-Related Side Effects and Adverse Reactions' OR 'Prescription Drugs/Adverse Effects'). The term 'pharmacovigilance' is also not included in the abstract or MeSH terms of Lu et al., and our search strategy was specific to the pharmacovigilance domain. We therefore cannot be certain that other relevant studies may have been missed although we believe a sufficient number of impact studies were identified to generalize our findings on the methodological approaches used and the few studies measuring impact on public health outcomes. We fully agree with the viewpoint of Christine Lu and Stephen Soumerai that impact research should apply rigorous study designs that can produce unbiased results and ITS is undisputedly one of the strongest quasi-experimental designs for evaluating pharmacovigilance regulatory interventions. We are pleased to have initiated a discussion on this relatively unexplored area and we thank Christine Lu and Stephen Soumerai for their constructive comments that will support further research in this field. There are no competing interests to declare.