The Curse of Cultural Recursion: Quantifying "Model Collapse" and Compounding Forgetting in Human Cultural Transmission

Overview This dataset and software package accompany the study "The Curse of Cultural Recursion," which hypothesizes that human culture exhibits "model collapse"—a phenomenon previously identified in recursive AI training—when subjected to centralized, high-volume transmission mechanisms. The research demonstrates that as cultural transmission becomes centralized, the overall complexity (Shannon Entropy) remains stable or increases, but the "long tail" of rare, niche, and artisanal motifs is exponentially erased. Methodology & Key Findings The study analyzes four disparate cultural domains across two centuries: Poetry (1800–1920) via Project Gutenberg. Popular Music Lyrics (1965–2023) via the Genius API. Material Culture/Headwear (1800–1950) via the Met Museum Open Access dataset. Political Rhetoric (1800–2024) via US State of the Union addresses. Using rarefaction to eliminate corpus-size bias, the analysis calculates Tail Retention Rates and Rolling Vocabulary Turnover. The data reveals a universal signature of cultural collapse: tail retention plummets to between 22% and 35% across all domains. Furthermore, a 200-year longitudinal analysis of political rhetoric identifies severe inflection points of "forgetting" in the 1900s and 1960s, demonstrating a mathematically compounding erosion of historical tradition. This cultural forgetting exhibits a statistically significant positive correlation with US urbanization rates (r=0.597,p=0.004r=0.597,p=0.004). Archive Contents This Zenodo repository contains the complete computational pipeline and data required to reproduce the study: *.py Scripts: Python scripts for data acquisition, NLTK preprocessing, Shannon entropy calculation, rarefaction, tail extraction, and longitudinal inflection point mapping. *_index.tsv Files: Metadata indices for all four domains, including document IDs, years, centralization labels, word counts, and file paths. *_raw/ Folders: The raw, preprocessed text files for Poetry, Headwear, and Speeches. Analysis Outputs: Terminal logs of the primary analysis, the extracted "Lost Tails" lists, and the generated longitudinal correlation graphs. Reproducibility & Copyright Notes Public Domain Data: The Poetry, Speeches, and Headwear raw text files are derived from Public Domain or CC0 sources (Project Gutenberg, US Government archives, Met Museum Open Access) and are included in this repository. Song Lyrics (Copyright Restriction): Raw lyric text is omitted from the songs_raw/ directory due to copyright restrictions. However, this domain is fully reproducible. Researchers can fetch the lyrics locally by obtaining a free API token from Genius, setting it in their environment, and running the provided songs_script.py. Dependencies: Requires Python 3.10+, pandas, nltk, requests, beautifulsoup4, matplotlib, and scipy. Declaration of AI-Assisted Research This research was conceptualized, designed, and interpreted by the author. The computational pipeline, data acquisition scripts, statistical modeling, and drafting of the methodology were developed with the assistance of a Large Language Model (LLM) strictly as a computational tool. All hypotheses, qualitative interpretations, methodological safeguards, and final conclusions are solely the work of the author.