A systematic review: Tools for assessing methodological quality of human observational studies

This review is, in press, to be published in the Australian National Health and Medical Research Council’s (NHMRC) Guidelines for Guidelines. Background: Observational studies are the primary human study type for evaluating potential harm from environmental exposures, and therefore critical for developing public health guidance. Although there are tools for assessing methodological quality in observational studies, there is no consensus on optimal tools or key elements to include when assessing quality. This study aims to help decision makers in selecting tools to assess risk of bias in observational studies of exposures. Objective: To identify, describe, categorize elements into domains, and evaluate published tools designed to assess methodological quality or risk of bias in observational human studies assessing the effects of exposures. Study appraisal and synthesis: Data was extracted on the characteristics, development and testing of each tool. The categories of items were classified into 9 domains informed by the literature. Results: We identified 62 tools with 17 categories of similar or overlapping items. Conclusions: Our review highlights the need for a common tool for assessing risk of bias in human observational studies of exposures. Absent that common tool, this review provides a clear cross-walk across available tools and identifies four main take home messages: (1) the tool should have clear definitions for each item and be transparent regarding the empirical or theoretical basis for each domain, (2) tools should include questions addressing 9 domains: Selection, Exposure, Outcome assessment, Confounding, Loss to follow-up, Analysis, Selective reporting, Conflicts of interest and Other, (3) the ratings for each domain should be reported, rather than an overall score, (4) the tool should be rigorously and independently tested for usability and reliability. Systematic Review Registration: PROSPERO: CRD42018094173