Benefits and strengths of the disproportionality analysis for identification of adverse drug reactions in a pharmacovigilance database

Adverse drug reactions (ADRs) represent an important medical issue: they result in 3–7% of all hospital admissions and are associated with a substantial increase in morbidity and mortality. Numerous methods can be used to investigate ADRs. Each has its strengths and weakness. The insufficiencies of basic (experimental) pharmacology as well as clinical trials for studying ADRs are well known. Animal physiology often differs from that of humans. Clinical trials, although necessary, do not allow definite conclusions, because they are built to evaluate efficacy more than safety. Thus, spontaneous notifications remain the cornerstone for ADRs despite their mandatory limitations (under-reporting, selective reporting, lack of denominator etc.). Intensive studies could allow quantification of a specific problem of drug safety (i.e. drug admission into hospital for ADRs). For some years, several pharmacoepidemiological methods have been used to identify and also to quantify ADRs. Among thesse methods, case–control or cohort studies are most widely used. However, their setting up requires specific organization, delay, money and also use of large databases, which are not often specifically built for evaluation of ADRs. In the present issue of the journal, clinical pharmacologists and pharmacoepidemiologists from Poitiers University Hospital (France) have used another method, working from the French PharmacoVigilance database. They used disproportionality analysis for identification of memory disorders associated with drugs 1. The present paper discusses the benefits and strengths of this method. As far as we know, the first time this approach was used was in the early 1980s in the field of drug safety during pregnancy. The question of a possible relation between valproic acid and spina bifida aperta was investigated by Robert 2 after a cluster of case reports in France from the Birth Defects Monitoring system in the Rhone-Alpes region (East France). Robert compared exposure to valproic acid in 146 women with infants suffering from spina bifida aperta and in 6616 other mothers with infants suffering from different malformations. They found a strong significant association, with an odds ratio of 20 and a P value <0.00001. This approach, called ‘case–control study’ in the original paper, was really the first performed case–noncase study. Following this first signal, further studies confirmed the teratogenicity of valproic acid in the first trimester of pregnancy. The second historical example was the investigation of a potential higher risk of serum-sickness-like syndrome related to cefaclor 3. After the report of several cases in the early 1990s, Stricker and Tijssen performed a ‘nested case–control study’ in the WHO Collaborating Centre for International Drug Monitoring Database, including reports from the USA, UK, Sweden, Canada and Germany for the period 1968–1987. They defined an ADR reporting odds ratio (ROR) as the ratio of the odds of the number of ADR reports of serum sickness in relation to cefaclor and amoxicillin or cephalexin and the odds of other reports on the same drugs. Using this case–noncase approach, they were able to identify that cefaclor had a significantly higher risk for serum-sickness-like syndrome compared with the two other antibiotics. During the 1990s, the databases collecting suspected ADR reports had grown, reaching sizes of more than several thousands or millions, thus making routine and regular quantitative screening a necessity. This data mining in these large databases has become a necessity in order to help pharmacovigilance systems to identify early signals for specific ADRs. Various statistical measures have been proposed for the application of computer-assisted quantitative signal detection procedures. Data mining encompasses a number of statistical techniques, including cluster analysis, link analysis, deviation detection and disproportionality assessment, which can be used to determine the presence of and to assess the strength of ADR signals. The use of a measure of disproportionality is currently applied in various national spontaneous reporting centres, as well as in the WHO Monitoring Centre. Several point estimates, such as ROR and proportional ADR reporting ratio (PRR), have been proposed, in order to analyse, for example, the UK Yellow Card Scheme spontaneous reporting database. Furthermore, the chance of the number of reports being reported in a certain combination with the assumption that no relation exists between the reported suspected ADR and the suspected medication can be calculated by means of the Poisson probability. Another approach is the use of Bayesian logic, specifying the relation between the prior and posterior probability before and after linking data fields, and of adding new data to the database. This method is currently used, for example by the WHO Monitoring Centre in the Bayesian confidence propagation neural network analysis (BCPNN). The statistical measures of disproportionality all express the extent to which the reported ADR is associated with the suspected drug compared with the other drugs in the database. The occurrence of ADRs related to other drugs in the database is used as a proxy for the background incidence of ADRs, when calculating the PRR. Calculations of measures of disproportionality are based upon a two-by-two contingency table (Table 1). Since all the measures of disproportionality are based on the same principles of calculation using the two-by-two table, results should be closely concordant. The abundant literature on this topic underlines that several quantitative methods are now available. However, none is universally better than the others. The different measures of disproportionality have both advantages and disadvantages, and the good choice relies upon on the specific data set and the aims of the screening. In a paper comparing the advantages of the ROR over the PRR, Rothman et al. 4 argued that the best way to deal with the weakness of the spontaneous reporting database should be to treat the data as source data for a case–control study. In this way, this ‘case–noncase’ approach focuses on the importance of the judicious choice of controls for the comparison, and highlights the inherent weakness in spontaneous reporting rata, which is very important for an optimal interpretation of such results. Disproportionality analysis is quick and inexpensive and, with some important precautions, is able to give valuable information on ADRs and drug safety. The main use for this method is to confirm (or not) a potential association based on a pharmacological hypothesis between a specific drug and an ADR. This can be illustrated by the relation between pioglitazone and bladder cancer. Experimental data support a potential association, because glitazones are peroxisome proliferator-activated receptor (PPAR) agonists. Studies in rats exposed to PPAR agonists indicated that tumours occurred in several tissues, with a distribution similar to that of PPARs. Clinical data (clinical trials and case reports) suggested the same association. Finally, a recent case–noncase study in the Food and Drug Administration (FDA) Adverse Event Reporting System (AERS) between 2004 and 2009 found a significant risk for pioglitazone (ROR = 4.30; 95% confidence interval 2.82–6.52). The risk of bladder cancer was less consistent for other hypoglycaemic drugs 5. In this situation of a pharmacological hypothesis, the advantage of the disproportionality method is the speed of its implementation. For example, we were able to demonstrate that rofecoxib exposure was significantly associated with a high ROR value (4.2) for thrombotic ADRs in the French PharmacoVigilance database, as early as the end of 2001, i.e. only 19 months after rofecoxib marketing 6. Rofecoxib was only withdrawn from the market in September 2004. Another example of this method to quickly investigate a health problem involving drugs could be severe necrotizing soft-tissue infections and nonsteroidal anti-inflammatory drugs. The case–noncase analysis performed in the French PharmacoVigilance database 7 led to the same conclusions as a case–control study, a method more expensive, longer and more difficult to perform. Thus, this method could be proposed, in a context of signal detection, for testing a working hypothesis before performing larger pharmacoepidemiological studies (case–control or cohort studies). Another important use for this method is validation of a pharmacological hypothesis about the mechanism of occurrence of ADRs. A nice example was the investigation of anti-human ether-a-go-go-related gene (HERG) activity of 52 proarrhythmic drugs and the risk of drug-induced arrhythmias 8. The study was performed using the database of the WHO International Drug Monitoring Program. A positive association between anti-HERG activity and the risk of severe ventricular arrhythmias and sudden deaths was found. Drugs which bind to HERG potassium channels in concentrations close to therapeutic plasma concentrations were shown to have a high risk of reports of serious ventricular arrhythmias and sudden deaths. This finding facilitates understanding of the mechanism of occurrence of such effects in humans, and could help to predict proarrhythmic effects of drugs by defining the value of preclinical HERG testing. A third application of the disproportionality methods concerns rare and/or nonspecific ADRs. For example, dilated cardiomyopathy is a common disease, which can be either idiopathic or the result of several aetiologies: genetic, viral or immune. Less frequently, it could be related to some toxic agents or drugs. A case–noncase study using the French PharmacoVigilance database was able to describe an association with some already suspected drugs (such as anthracyclines and antiretrovirals), but also with other drugs (antipsychotics, lithium, antidepressants and retinoids) less known to induce such an ADR 9. This finding represents a pharmacovigilance signal which needs to be investigated by future prospective studies. Memory disorders are another example of a nonspecific ADR with several other nonpharmacological explanations. Thus, the impact of drugs in memory disorders could be minimized. The case–noncase study published by the French team of pharmacoepidemiology from Poitiers University Hospital in the present issue and performed using the French PharmacoVigilance database confirmed an association between memory disorders and some drugs, such as benzodiazepines or anticonvulsants, but also found an association with other drugs, such as ‘benzodiazepine-like’ hypnotics (zolpidem and zopiclone), newer anticonvulsants, serotoninergic antidepressants, isotretinoin or cyclosporine 1. The final application of disproportionality methods could be to generate automatic signals from large postmarketing or pharmacovigilance databases. As the complete safety profile of a drug can be described only after its marketing approval, surveillance systems are needed, and suspected ADRs are now collected in very large databases. As the volume of these data is continuously growing, data mining with measures of disproportionality is being used more and more in order to detect new, previously unknown, ADRs as soon as possible after a drug is marketed. For example, such a method was used to detect the pregabalin dependence potential in the Swedish national register of adverse drug reactions (SWEDIS) 10. This study led to the addition of warnings on pregabalin abuse potential in the Summary of Product Characteristics in April and June 2010. However, this conclusion should be challenged. Several experiments have recently shown clearly that use of this approach to automatically generate signals on drug safety is not always satisfactory. In fact, a case–noncase analysis necessitates a double analysis: firstly, a pharmacological one and secondly, a medical one. All disproportionality analysis in a pharmacovigilance database requires a clear pharmacodynamic hypothesis established on basic properties of drugs. Several examples are given in the present paper, such as the carcinogenic properties of pioglitazone on bladder 5, the thrombotic risk of coxibs 6 and the involvement of nonsteroidal anti-inflammatory drugs in necrotizing soft-tissue infections 7. This case–noncase method cannot be used for investigating all risks of all drugs without a strong basic pharmacodynamic hypothesis. For example, a systematic investigation of a cancer or depression risk for drugs (whatever their class) using this approach in such a database without a biological hypothesis has no sense and could lead (and does lead) to false findings. Another comment about this disproportionality analysis concerns validation of cases. In order to obtain valid results, it is necessary that, before analysis, each ADR report is validated once again in the context of the pharmacological and medical questions of the study. If not, it could induce false results without clinical meaning. Finally, it is important to recall that these disproportionality studies should be only considered as exploratory in a context of signal detection. They do not allow quantification of the true risk. For example, ROR only investigates an increased risk of ADR reporting and not risk of ADR occurrence in absolute terms. Despite its inherent limits, disproportionality analysis in pharmacovigilance databases is now a validated method in drug safety research and surveillance, although this kind of approach should only be considered as exploratory to generate signals. Finding of a disproportionality ratio for a drug should lead to a new reinvestigation of data from experimental pharmacology and randomized clinical trials. It should also stimulate specific case–control or cohort analysis to confirm the signal. This paper clearly underlines that none of the methods described above and taken alone (experimental data, clinical trials, spontaneous notifications, case–control studies, cohort studies and data mining) should be considered as definitive for evaluating drug risk. It is only the convergence of proofs which allows final conclusions and decisions in pharmacovigilance. Thus, the notion of ‘levels of evidence’, widely used for evaluating drug efficacy, cannot be applied in the field of ADRs; all methods are of interest for evaluation of ADRs. Finally, the ease with which disproportionality studies can be performed appears to be important today, when there is a growing demand for more safe drugs. There are no competing interests to declare.