Answers are grounded in this paper and cite their sources.
Background/Objectives: This observational study assessed rating agreement within the non-blinded Ágora nutrition-information verification platform. Methods: We analyzed 13,709 records for 791 Instagram publications from 286 evaluator identifiers. Publication dates ranged from 21 October 2023 to 14 June 2026 and rating dates from 25 October 2023 to 16 July 2026. Analyses included descriptive dispersion, crossed Gaussian REML variance decomposition with 500 parametric bootstrap replicates, cumulative-logit sensitivity, ordinal Krippendorff alpha, duplicate-pair selections and evaluator-influence restrictions. Comment comparisons were descriptive. Results: Ratings were polarized, with 88.7% (scores 0–1 or 4–5). The median within-publication sample was 0.47; 52.3% had SD < 0.5 and 6.6% had SD ≥ 1.5, interpreted as bounded-scale dispersion thresholds. Gaussian variance proportions were 86.45% for publications (95% interval 85.15–87.57%), 0.73% for evaluators (0.53–0.96%) and 12.82% for residual variation (11.77–14.02%). The ordinal model preserved this ordering on its latent scale. Ordinal alpha was 0.8457–0.8462 across distinct-pair selections. The broad decomposition persisted after excluding the most active 58 evaluators and in five capped-contribution samples, with reduced publication coverage. Intermediate ratings more often included comments (85.1% vs. 67.1%; descriptive OR 2.79). Conclusions: These results describe observed within-platform agreement; visible prior ratings prevent distinguishing independent agreement from convergence. Factual validity and clinical benefits were not evaluated. Ágora confirmed eligibility of all evaluators. Repeated export records were addressed through alternative-record sensitivity analyses.
No takes yet. Share an insight, caveat, or question.
Frauca et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: