EudraScore: Modification of the Vigi4Eudra-score for the Assessment of the Completeness of Adverse Drug Reaction Reports in EudraVigilance

In the absence of a gold standard for assessing the completeness of spontaneous adverse drug reaction (ADR) reports, most studies have relied on assessing the proportion of reports containing information on predefined categories. The same principle applies to the Vigi4Eudra-score, which can be used to analyse the completeness of ADR reports from EudraVigilance. Moreover, ADR reports with a high level of information provided may be more useful in pharmacovigilance practice. The first aim of our study was to further develop the already existing Vigi4Eudra-score by modifying its penalties and by adding additional relevant categories (= EudraScore). The second aim was to assess whether the EudraScore performed better than the Vigi4Eudra-score to identify reports with at least a possible causal relationship as “well documented”. An expert panel was established comprising assessors from the pharmacovigilance and the research department of the German Federal Institute for Drugs and Medical Devices, with many years of experience in the assessment of ADR reports. Based on expert consensus, the Vigi4Eudra-score was modified by adapting existing categories and incorporating additional causality-relevant criteria, resulting in the EudraScore. A use case was conducted to compare the values calculated using the EudraScore with those obtained from the Vigi4Eudra-score. The performance of both scores in identifying reports with an “at least possible” causal relationship as “well documented” was evaluated by calculating sensitivity and specificity. Compared to the Vigi4Eudra-score, some penalties were adjusted for the EudraScore, with reduced penalties for missing country of origin and drug dose (from 10 to 5%) and an increased penalty for the absence of a free-text description (from 10 to 30%) when information on time to onset and ADR outcome is missing. The categories patient’s history (+ 10%), concomitant drugs (+ 10%), rechallenge/dechallenge/action taken with the drug (+ 10%), and seriousness (+ 5%) were added as a bonus score. Finally, the bonus score was weighed by the respective penalty score. In the use case, most reports received higher values with the EudraScore compared to the Vigi4Eudra-score, although the differences were generally small (≤ 0.1). The categories drug dose, seriousness, and patient’s history mostly led to the observed differences. Although the completeness does not directly reflect the strength of the causal association, the EudraScore received a higher sensitivity (0.347 [0.293–0.405]) compared to the Vigi4Eudra-score (0.245 [0.197–0.299], p < 0.001) and a negligibly lower specificity (EudraScore: 0.907 [0.860–0.940], Vigi4Eudra-score: 0.932 [0.889–0.959], p = 0.07) for identifying reports with an “at least possible” causal relationship as “well documented”. Although the sensitivity was rather low, the EudraScore appears to be more suitable for the prioritisation of ADR reports of higher relevance than the Vigi4Eudra-score. Additionally, further development of the EudraScore to not only assess the presence of information but also information quality could add substantial value.