Quality of reporting of randomised controlled trials of artificial intelligence in healthcare: a systematic review

OBJECTIVES: The aim of this study was to evaluate the quality of reporting of randomised controlled trials (RCTs) of artificial intelligence (AI) in healthcare against Consolidated Standards of Reporting Trials-AI (CONSORT-AI) guidelines. DESIGN: Systematic review. DATA SOURCES: We searched PubMed and EMBASE databases for studies reported from January 2015 to December 2021. ELIGIBILITY CRITERIA: We included RCTs reported in English that used AI as the intervention. Protocols, conference abstracts, studies on robotics and studies related to medical education were excluded. DATA EXTRACTION: The included studies were graded using the CONSORT-AI checklist, comprising 43 items, by two independent graders. The results were tabulated and descriptive statistics were reported. RESULTS: We screened 1501 potential abstracts, of which 112 full-text articles were reviewed for eligibility. A total of 42 studies were included. The number of participants ranged from 22 to 2352. Only two items of the CONSORT-AI items were fully reported in all studies. Five items were not applicable in more than 85% of the studies. Nineteen per cent (8/42) of the studies did not report more than 50% (21/43) of the CONSORT-AI checklist items. CONCLUSIONS: The quality of reporting of RCTs in AI is suboptimal. As reporting is variable in existing RCTs, caution should be exercised in interpreting the findings of some studies.