Automating risk of bias assessment for clinical trials

In medicine, the publication of clinical trials now far out-paces clinicians' ability to read them. Systematic reviews, which aim to summarize the entirety of the available evidence on a specific clinical question, have therefore become the linchpin of evidence-based decision making. A key task in systematic reviews is determining whether the results of included studies may be affected by biases, e.g., poor randomization or blinding. This is called risk of bias assessment and is now standard practice. Standardized tools are used to perform these assessments; a notable example being the Cochrane risk of bias tool, which covers seven different types of potential biases and involves researchers extracting sentences from articles to support their bias assessments. These assessments are crucial in interpreting published evidence, but due to the exponential growth of the biomedical literature base, manually assessing the risk of bias in clinical trials has grown burdensome for clinical researchers. Aiming to mitigate this workload, we explore automating risk of bias assessment. We demonstrate that systematic reviews may be used to distantly supervise text mining models, obviating the need for manually annotated clinical trial reports. Specifically, we leverage data from the Cochrane Database of Systematic Reviews (a large repository of systematic reviews), and link clinical trial reports to structured data from the same studies found in CDSR to produce a pseudo-annotated labeled corpus. We then develop a joint model which, using (the PDF of) a clinical trial report as input, predicts the risks of bias in each of the aforementioned seven areas while simultaneously extracting the text fragments supporting these assessments.