Mining the Crowd: A Structural Diagnostic Framework for Evaluating X's Community Notes

Community Notes presents moderation as a crowd judgment problem, but whether the crowd actually functions as a credible decision-making system is an empirical question. This thesis introduces the term structural accountability as an analytical concept for evaluating whether the internal organization of a participatory moderation system supports interpreting its outcomes as the result of distributed collective judgment. It then uses that concept to assess the extent to which X's Community Notes produces collective-intelligence outcomes through a structurally accountable decision process. Using the public Community Notes data release from 2021 to 2025, restricted to English-language notes for semantic analysis and linked to ratings and note-level outcomes, the analysis develops four structural diagnostics: participation diversity, decentralization of evaluative labor, independence of rater-account behavior, and aggregation consistency. The results show that Community Notes is broad in participation but highly unequal in practical contribution, with evaluative labor concentrated in a relatively small subset of rater accounts, especially in politically salient domains. At the same time, long-run behavioral similarity among rater accounts remains low, indicating that observed rating profiles do not generally collapse into a small set of interchangeable evaluative patterns. The strongest structural constraint appears in aggregation; the estimated input-to-outcome mapping is highly conservative, and comparable rating inputs produce different outcome probabilities across topics. These findings show that Community Notes satisfies some conditions associated with collective intelligence more clearly than others. The thesis contributes a framework for evaluating participatory moderation systems through their internal structure and shows that claims about crowd judgment require empirical scrutiny of how participation is distributed, whether rating behaviors remain distinct, and how these behaviors are converted into visible outcomes.