Assessing the quality of reporting of harms in randomized controlled trials published in high impact cardiovascular journals

Randomized controlled trials (RCTs) and their systematic reviews have the highest impact in the hierarchy of research designs; however, RCTs are usually powered to evaluate therapeutic efficacy, rather than adverse events (AEs) of an intervention. Since AEs or harms can impact the quality of life, therapeutic adherence, and health status,1 adequate and transparent reporting of harms is critical to facilitating informed decision-making. Although variable and suboptimal adherence to the CONSORT harms checklist has been observed in various specialties, current compliance and quality of harms reporting among cardiovascular (CV) trials is unknown. We determined adherence of contemporary RCTs to the CONSORT extension for harm reporting checklist in three CV journals. In October 2018, we conducted a descriptive, cross-sectional study using PubMed/MEDLINE to identify all RCTs published between 2011 and 2017, in three CV journals: Circulation, European Heart Journal (EHJ), and Journal of the American College of Cardiology (JACC). The journals were selected based on high impact factor, h-5 index, and their sole focus on publishing trials in CV medicine. According to the endorsers’ section on the CONSORT statement website, JACC and Circulation have endorsed CONSORT. The EHJ has recommended following CONSORT guidelines for the reporting of clinical trials under the instructions to authors section. The data were collected according to the recommendations laid down by CONSORT for harms extension. We determined adherence of RCTs to the CONSORT for Harms Guidelines by calculating overall proportion (%) of RCTs that included each of the individual items included in the checklist followed by an overall mean. Additionally, we used the binomial distribution for all checklist items to determine confidence intervals for their means and the Pearson Correlation to test year of publication and overall compliance with the checklist for individual items. Finally, we employed the Kruskal–Wallis test to determine whether types of funding sources had an impact on the overall adherence of the trials. Our final cohort for analysis included 153 two-parallel-armed RCT abstracts. The detailed selection process is shown in Figure 1. Of the 153 trials, 54.9% (n = 84/153) were European in origin, 42.5% (n = 65/153) focused on preventive cardiology, and almost all (95.4%, n = 146/153) declared funding. More than half (57.5%; n = 88/153) specified the definition of AEs, while most trials (80.4%; n = 123/153) mentioned all AEs in methods. Approximately two-thirds (62.1%; n = 95/153) of the trials reported treatment-emergent AEs and all AEs, while almost 70% of the trials reported serious AEs and deaths in each intervention arm. One-third (32.7%; n = 50/153) of trials noted the use of a validated instrument to report AEs. Most trials (85.6%, n = 131/153) reported the time frame when harms were collected, and about half (52.3%; n = 80/153) described the attribution of the AEs to the specific intervention of interest. Many trials (59.5%; n = 91/153) also described presentation and analysis of harms. Only a few trials (n = 39/153) mentioned dictionary definitions for coding AEs and described how the recurrent AEs were handled (n = 29/153). The overall adherence to the CONSORT Extension for Harms statement was 59.3% (confidence interval 0.51–0.67), while the individual adherence between the journals varied (Table 1). Furthermore, of the 153 RCTs, 16 RCTs had more than 1000 participants. Four of the checklist items: time frame, denominators for analysis, separate results for each treatment group, and the number of events and patients with events were reported by all the large-scale RCTs (n = 16/16). Flow diagram of the study. Flow diagram of the study. Reporting of AEs in RCTs as per the CONSORT Extension for Harms statement Recommendation 9 was not included in the final analysis as none of the 153 trials looked at subgroup. AE, adverse event; CONSORT, Consolidated Standards of Reporting Trials; RCT, randomized controlled trial. Reporting of AEs in RCTs as per the CONSORT Extension for Harms statement Recommendation 9 was not included in the final analysis as none of the 153 trials looked at subgroup. AE, adverse event; CONSORT, Consolidated Standards of Reporting Trials; RCT, randomized controlled trial. In this contemporary assessment of 153 RCTs, we found that the overall adherence to CONSORT Harm reporting was 59%. Over the study period (2011 to 2017), we did not observe a substantial improvement in adherence. Even among select trials of larger size, reporting on all elements of harms reporting was incomplete, and adherence only reached 75%. These data do suggest that studies with large sample sizes tend to have better reporting quality, perhaps due to greater access to resources.2 Moreover, it has been suggested that AE reporting in RCTs and compliance with CONSORT guidelines remains low mainly due to restrictions on manuscript length3 and a lack of enforcement by journals themselves.4 Authors frequently deem AE reporting as insignificant,5 which may explain why 15 years after the establishment of the CONSORT extension for harms, non-compliance remains high. That being said, we did observe an improvement of the reporting of various items, such as descriptions for presenting and analysing harms, denominators for analyses, separate AEs, serious AEs and deaths in each arm, and discussion of both efficacy and harms from 2011 to 2017. Furthermore, reporting of certain items such as the time frame of harms-related data collection and number of events and number of patients with events show comparatively better reporting (>80%) which suggests that authors place varying levels of value on certain items as opposed to others. Our data, therefore, suggest the need for standardization and more rigorous enforcement of the CONSORT guidelines by journals, editors, and reviewers to ensure high-quality reporting of safety data, as enforcement has shown to improve adherence in other CONSORT domains such as abstract reporting. This study has certain limitations. First, only three journals were selected. Second, a vast majority of selected RCTs were from JACC, which might have skewed the results. Third, each item was given equal weight, which might not be appropriate for certain items, such as the use of a validated instrument and reasons for treatment withdrawal. Finally, as the CONSORT statement does not take the trial registration into account, we did not examine registries for harms-related data; however, all available supplementary material was assessed. In conclusion, aggregate adherence to CONSORT harms guidelines in RCTs published in three prominent CV journals is suboptimal and has not substantially improved over time. There is an unmet need for enforcement of these guidelines by journal editors and editorial boards. Adherence to CONSORT harms reporting guidelines allows for a more balanced perspective of the efficacy and safety of investigational therapies.