Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Extracting memorized pieces of (copyrighted) books from open-weight language models

A. Feder Cooper, Aaron Gokaslan, Ahmed Ahmed, Amy B. Cyphert, Christopher De Sa, Mark A. Lemley, Daniel E. Ho, Percy Liang

TL;DR

This work develops a probabilistic, sliding-window framework to quantify verbatim memorization of copyrighted books in open-weight LLMs, revealing substantial variation across books and models. By computing the suffix probability p_z for 50-token suffixes within 100-token sequences and treating p_z as extraction risk, the study shows that most models memorize only small fractions of text, yet certain books (notably Harry Potter and 1984) are memorized in whole by some models like Llama 3.1 70B, enabling near-verbatim reconstruction with seeds. The results have nuanced copyright implications: memorization can yield derivative-like copies and, in some cases, practically infringing outputs, though class-action claims may be difficult to certify due to case-by-case variability. The authors validate their method with longer prefixes, negative controls, and cross-model analyses, and discuss policy considerations and future directions for assessing memorization and its legal consequences.

Abstract

Plaintiffs and defendants in copyright lawsuits over generative AI often make sweeping, opposing claims about the extent to which large language models (LLMs) have memorized plaintiffs' protected expression in their training data. Drawing on both machine learning and copyright law, we show that these polarized positions dramatically oversimplify the relationship between memorization and copyright. To do so, we extend a recent probabilistic extraction technique to measure memorization of 50 books in 17 open-weight LLMs. Through thousands of experiments, we show that the extent of memorization varies both by model and by book. With respect to our specific extraction methodology, we find that most LLMs do not memorize most books -- either in whole or in part. However, we also find that Llama 3.1 70B entirely memorizes some books, like the first Harry Potter book and 1984. In fact, the first Harry Potter is so memorized that, using a seed prompt consisting of just the first few tokens of the first chapter, we can deterministically generate the entire book near-verbatim. We discuss why our results have significant implications for copyright cases, though not ones that unambiguously favor either side.

Extracting memorized pieces of (copyrighted) books from open-weight language models

TL;DR

This work develops a probabilistic, sliding-window framework to quantify verbatim memorization of copyrighted books in open-weight LLMs, revealing substantial variation across books and models. By computing the suffix probability p_z for 50-token suffixes within 100-token sequences and treating p_z as extraction risk, the study shows that most models memorize only small fractions of text, yet certain books (notably Harry Potter and 1984) are memorized in whole by some models like Llama 3.1 70B, enabling near-verbatim reconstruction with seeds. The results have nuanced copyright implications: memorization can yield derivative-like copies and, in some cases, practically infringing outputs, though class-action claims may be difficult to certify due to case-by-case variability. The authors validate their method with longer prefixes, negative controls, and cross-model analyses, and discuss policy considerations and future directions for assessing memorization and its legal consequences.

Abstract

Plaintiffs and defendants in copyright lawsuits over generative AI often make sweeping, opposing claims about the extent to which large language models (LLMs) have memorized plaintiffs' protected expression in their training data. Drawing on both machine learning and copyright law, we show that these polarized positions dramatically oversimplify the relationship between memorization and copyright. To do so, we extend a recent probabilistic extraction technique to measure memorization of 50 books in 17 open-weight LLMs. Through thousands of experiments, we show that the extent of memorization varies both by model and by book. With respect to our specific extraction methodology, we find that most LLMs do not memorize most books -- either in whole or in part. However, we also find that Llama 3.1 70B entirely memorizes some books, like the first Harry Potter book and 1984. In fact, the first Harry Potter is so memorized that, using a seed prompt consisting of just the first few tokens of the first chapter, we can deterministically generate the entire book near-verbatim. We discuss why our results have significant implications for copyright cases, though not ones that unambiguously favor either side.
Paper Structure (146 sections, 18 equations, 122 figures, 3 tables)

This paper contains 146 sections, 18 equations, 122 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Quantifying memorization through extraction
  3. Probabilistic discoverable extraction
  4. Interpreting target suffix probabilities as extraction risk
  5. Memorization, extraction risk, and U.S. copyright
  6. Measuring memorization of individual books
  7. Sliding-window probabilistic extraction procedure
  8. Average extraction rates obscure book-specific memorization
  9. Validating our extraction procedure
  10. Longer prefixes surface more memorized training data
  11. Measurements on non-training data do not mistakenly register extraction
  12. Implications for measuring probabilistic extraction on books
  13. Comparing memorization across books and models
  14. Experimental setup
  15. Key patterns in our results
  16. ...and 131 more sections

Figures (122)

  • Figure 1: Generating the exact completion of a quote from The Great Gatsbygatsby with Llama 1 30B.
  • Figure 2: Plotting $p_{\bm{z}}$ for the "careless people" quote from The Great Gatsby (Figure \ref{['fig:header']}).
  • Figure 3: Visualizing results of the sliding-window probabilistic extraction procedure. For each $100$-token window ($50$-token prefix $+$$50$-token suffix) across 19841984, we compute $p_{\bm{z}}$, the probability that Llama 3.1 70B generates the exact $50$-token suffix given the $50$-token prefix, with respect to top-$k$ decoding ($T{=}1$, $k{=}40$). (a) Scatterplot of all extracted sequences ${\bm{z}}$ shown by their start position in the book. The $100$-token sequences overlap significantly. (b) Condensed heatmap view. At each character position, we plot the maximum$p_{\bm{z}}$ across all overlapping sequences whose suffix covers that position. In both plots, colors encode $p_{\bm{z}}$ on a $\log$ scale. We deem extraction successful if $p_{\bm{z}}\!\geq\!\tau_\text{min}\!=\!0.1\%$, considering values below this to not reflect extraction (i.e., rounding $p_{\bm{z}}$ down to $0\%$). See Appendix \ref{['app:sec:sliding-window']}.
  • Figure 4: Average extraction rates are low, but book-specific extraction varies widely. (left) Comparing extraction rates (Equation \ref{['eq:rate:main']}) of random $100$-token sequences ($50$-token prefix $+$$50$-token suffix) from Books3 for different models trained on Books3. We show the greedy (blue) and probabilistic (orange, $p_{\bm{z}}\!\geq\!\tau_{\text{min}}\!=\!0.1\%$, top-$k$ decoding with $T{=}1$ and $k{=}40$) discoverable extraction rates (Appendix \ref{['app:sec:rates']}). For both, average extraction is low. (right) Extraction coverage (Equation \ref{['eq:cov']}) for specific books reveals a more nuanced picture. For Harry Potter and the Sorcerer's Stonehp1 and Sandman Slimsandmanslim, we show coverage for $\tau \in \{1\%, 10\%, 50\%, 75\%\}$ for $3$ models (Appendix \ref{['app:sec:sliding-window:percentage']}). Nearly half of Harry Potter and the Sorcerer's Stone can be extracted with respect $p_{\bm{z}}\!\geq\!\tau\!=\!50\%$.
  • Figure 5: Extraction coverage (Equation \ref{['eq:cov']}) differs across models for the $50$ books we evaluate, illustrated for Llama 1 65B and Llama 3.1 70B. Results use $100$-token sequences ($50$-token prefix $+$$50$-token suffix), top-$k$ decoding ($T{=}1$, $k{=}40$), and coverage threshold $p_{\bm{z}}\!\ge\!\tau\!=\!10\%$.
  • ...and 117 more figures

Theorems & Definitions (5)

  • Definition 1: $(n, p)$-discoverable extraction, from hayes2025measuringmemorizationlanguagemodels
  • Definition 1: Extraction coverage
  • Definition 1: Model Knowledge Extraction, from carlini2021extracting
  • Definition 2: $k$-Eidetic Memorization, from carlini2021extracting
  • proof