Critical appraisal and issues regarding generalisability of comparative effectiveness studies of NOACs in atrial fibrillation and their relation to clinical trial data: a systematic review

OBJECTIVE: To critically appraise the published comparative effectiveness studies on non-vitamin K antagonist oral anticoagulants (NOACs) in non-valvular atrial fibrillation (NVAF). Results were compared with expectations formulated on the basis of trial results with specific attention to the patient years in each study. METHODS: All studies that compared the effectiveness or safety between at least two NOACs in patients with NVAF were eligible. We performed a systematic literature review in Medline and EMbase to investigate the way comparisons between NOACs were made, search date 23 April 2019. Critical appraisal of the studies was done using among others ISPOR Good Research Practices for comparative effectiveness research. RESULTS: We included 39 studies in which direct comparison between at least two NOACs were made. Almost all studies concerned patient registries, pharmacy or prescription databases and/or health insurance database studies using a cohort design. Corrections for differences in patient characteristics was applied in all but two studies. Eighteen studies matched using propensity scores (PS), 8 studies weighted patients based on the inverse probability of treatment, 1 study used PS stratification and 10 studies applied a proportional hazards model. These studies have some important limitations regarding unmeasured confounders and channelling bias, even though the larger part of the studies were well conducted technically. On the basis of trial results, expected differences are small and a naïve analysis suggests trials with between 7200 and 56 500 patients are needed to confirm the observed differences in bleedings and between 51 800 and 7 994 300 to confirm differences in efficacy. DISCUSSION: Comparisons regarding effectiveness and safety between NOACs on the basis of observational data, even after correction for baseline characteristics, may not be reliable due to unmeasured confounders, channelling bias and insufficient sample size. These limitations should be kept in mind when results of these studies are used to decide on ranking NOAC treatment options.