Verbatim in the Weights: Quantifying Copyright Memorization Across Open-Weight Language Models

We present a large-scale empirical study quantifying the memorization of copyrighted material in open-weight language models. Analyzing 81 models across 12 organizations, we introduce three novel forensic methods: (1) *Dataset Fingerprinting*, which distinguishes memorization of specific text versions from general content familiarity by comparing model perplexity on original versus sentence-reordered passages; (2) a *Memorization Scaling Law* quantifying how copyright retention increases with model size across three independent model families; and (3) *Evasion Detection via Min-K% Divergence*, demonstrating that instruction-tuned models retain identical copyright knowledge as their base counterparts despite safety training. Our experiments reveal that leading models reproduce copyrighted book passages with up to 100% verbatim accuracy, that larger models memorize 10–14% more copyrighted content than their smaller siblings, and that the fingerprint ratio for Harry Potter in Qwen2.5-7B reaches 4.49×, meaning the model recognizes the exact published text nearly five times better than a content-equivalent paraphrase. These findings carry significant implications for copyright law, AI regulation, and the transparency obligations of model developers.