A meta-epidemiological study found lack of transparency and poor reporting of disproportionality analyses for signal detection in pharmacovigilance databases

OBJECTIVES: To review and appraise methods and reporting characteristics of pharmacovigilance disproportionality analyses. STUDY DESIGN AND SETTING: We randomly selected 100 disproportionality analyses indexed in Medline found during a systematic literature search. We then extracted and synthetized methodological and reporting characteristics using seven key items: (1) title transparency; (2) protocol pre-registration; (3) date of data extraction and analysis; (4) outcome, population, exposure and comparator definitions; (5) adjustment and stratification of results; (6) method and threshold for signal detection; (7) secondary and sensitivity analyses. RESULTS: We found that methods used to generate disproportionality signals were extremely heterogeneous; there were nearly as many unique analyses as studies. The authors used various populations, methods, signal detection thresholds, adjustment or stratification variables, generally without justification for their choice or pre-specification in protocols. Moreover, 78% of studies failed to report methods for case, adverse drug reactions or comparator selection and 32 studies did not define the threshold for signal generation. CONCLUSION: Our survey raises major concerns regarding all aspects of disproportionality analyses that could lead to misleading results and generate unjustified alarms. We advocate for a strong and transparent rationale for variable selection, choice of population and comparators pre-specified in a protocol and assessed by sensitivity analyses.