Objective To explore assessors' experience regarding the use of two approaches that evaluate the certainty of evidence from Network Meta-Analysis (NMA): Grading of Recommendations Assessment, Development and Evaluation (GRADE) and Confidence In Network Meta-Analysis (CINeMA).Study Design And Setting Thirteen assessors trained in NMA methodology and evaluation methods were randomly assigned to four groups to apply GRADE and CINeMA to two networks each to compare the approaches' concordance, inter-rater reliability and application time (Minozzi et al, 2025). Assessors' experiences were collected through an online questionnaire including open-ended questions. The questionnaire explored usability, strengths, weaknesses, and perceived accuracy of each approach. Each assessor was surveyed twice: once after GRADE use, and once after CINeMA use. Thematic analysis was used to analyze their responses.Results We identified seven themes: instruments' challenges, knowledge need, process execution, positive aspects, interpretative uncertainty, implementation, level of confidence. GRADE assessment was considered long because of the extensive material to manage and at high risk of subjectivity but with a transparent and clear process. CINeMA automatization allowed a quick and easy use but impacted negatively on the process methodological understanding. Technical challenges related to website and files management were identified. Overall, assessors highlighted the need for further training on tools' usage.Conclusion GRADE was considered time-consuming and at high risk of subjectivity but excelled in transparency and clarity. CINeMA was considered high-speed but showed some practical and output-interpretation challenges, particularly where specialized knowledge of NMA methods was required. Additional training courses were suggested for both tools.
Battain et al. (Tue,) studied this question.