Certainty of evidence and intervention's benefits and harms are key determinants of guidelines’ recommendations

OBJECTIVE: Many factors are postulated to affect guidelines developments. We set out to identify the key determinants. STUDY DESIGN AND SETTING: a) Web-based survey of 12 panels of 153 "voting" members who issued 2941 recommendations; b) qualitative analysis of 13 panels of 311 attendees (panel members, systematic review teams and observers). RESULTS: Compared with "no recommendations", when intervention's benefit outweigh harms (BH-balance), probability of issuing strong recommendations in favor of intervention was 0.22 (95%CI: 0.08 to 0.36) when certainty of evidence (CoE) was very low; 0.5 (95%CI:0.36 to 0.63) when low; 0.74 (95%CI 0.61 to 0.87) when moderate and 0.85 (95%CI:0.71 to 1.00) when high. No other postulated factor significantly affected recommendations. The findings are consistent with a J- curve model when recommendations are issued in favor but not against an intervention. Panelists often changed their judgments as a result of the meeting discussion (67% for CoE to 92% for balance between benefits and harms). The panels spent over 50% of their time debating CoE; the chairs and co-chairs dominated discussion. CONCLUSIONS: CoE and BH-balance are key determinants of recommendations in favor of an intervention. Chairs and co-chairs dominate discussion. Panelists often change their judgments as a result of panel deliberation.