Reliability of the risk of bias assessment in randomized controlled trials for nursing: A cross‐sectional study

AIM: To evaluate the percentage and reasons for disagreements in the risk of bias (RoB) assessments for randomized controlled trials (RCTs) included in more than one Cochrane review in the field of nursing. BACKGROUND: Disagreement in RoB assessments reduces the credibility of the evidence summarized by systematic reviews (SRs). There is no study that evaluates the reliability of RoB assessments in nursing studies. DESIGN: Secondary data analysis based on research reports. METHODS: RCTs included in more than one review in the nursing have been included. The disagreement of the assessment was analysed, and the possible reasons for disagreements were investigated. RESULTS: Twenty-three RCTs were included in more than one review. The agreement of assessment ranged from 36.84% for "selective reporting" to 91.30% for "random sequence generation". "Allocation concealment" showed the optimal agreement (84.21%). The items "blinding of participants and personnel", "blinding of outcome assessment" and "incomplete outcome data" showed poor agreement, with 50.00%, 58.82% and 66.67%, respectively. Most disagreements came from extracting incomplete or different RCTs' information. CONCLUSIONS: The level of agreement of the assessment between reviews has varied greatly in the field of nursing. More complete and accurate information of RCTs needs to be collected when conducting a SR.