Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Public Access in LLM Pre-Training Data

Sruly Rosenblat, Tim O'Reilly, Ilan Strauss

TL;DR

This paper investigates whether non-public, paywalled copyrighted content from O'Reilly Media was included in OpenAI's pre-training data by applying the DE-COP membership inference attack across GPT-3.5 Turbo, GPT-4o, and GPT-4o Mini to 13,962 paragraphs from 34 books. Using book- and paragraph-level AUROC metrics, the study finds that GPT-4o exhibits strong recognition of non-public content (book-level AUROC ≈ 0.82), GPT-3.5 Turbo remains near random (~0.50), and GPT-4o Mini also performs near random, suggesting a time-updated trend toward greater use of non-public data. The authors discuss robustness to temporal bias, acknowledge limitations in testing smaller models, and argue for greater transparency and licensing mechanisms to enable responsible data sourcing. The work highlights systemic data provenance issues in AI training, underscores the need for formal data licensing marketplaces, and calls for regulatory and industry standards to protect content creators while sustaining AI advancement.

Abstract

Using a legally obtained dataset of 34 copyrighted O'Reilly Media books, we apply the DE-COP membership inference attack method to investigate whether OpenAI's large language models were trained on copyrighted content without consent. Our AUROC scores show that GPT-4o, OpenAI's more recent and capable model, demonstrates strong recognition of paywalled O'Reilly book content (AUROC = 82\%), compared to OpenAI's earlier model GPT-3.5 Turbo. In contrast, GPT-3.5 Turbo shows greater relative recognition of publicly accessible O'Reilly book samples. GPT-4o Mini, as a much smaller model, shows no knowledge of public or non-public O'Reilly Media content when tested (AUROC $\approx$ 50\%). Testing multiple models, with the same cutoff date, helps us account for potential language shifts over time that might bias our findings. These results highlight the urgent need for increased corporate transparency regarding pre-training data sources as a means to develop formal licensing frameworks for AI content training

Beyond Public Access in LLM Pre-Training Data

TL;DR

This paper investigates whether non-public, paywalled copyrighted content from O'Reilly Media was included in OpenAI's pre-training data by applying the DE-COP membership inference attack across GPT-3.5 Turbo, GPT-4o, and GPT-4o Mini to 13,962 paragraphs from 34 books. Using book- and paragraph-level AUROC metrics, the study finds that GPT-4o exhibits strong recognition of non-public content (book-level AUROC ≈ 0.82), GPT-3.5 Turbo remains near random (~0.50), and GPT-4o Mini also performs near random, suggesting a time-updated trend toward greater use of non-public data. The authors discuss robustness to temporal bias, acknowledge limitations in testing smaller models, and argue for greater transparency and licensing mechanisms to enable responsible data sourcing. The work highlights systemic data provenance issues in AI training, underscores the need for formal data licensing marketplaces, and calls for regulatory and industry standards to protect content creators while sustaining AI advancement.

Abstract

Using a legally obtained dataset of 34 copyrighted O'Reilly Media books, we apply the DE-COP membership inference attack method to investigate whether OpenAI's large language models were trained on copyrighted content without consent. Our AUROC scores show that GPT-4o, OpenAI's more recent and capable model, demonstrates strong recognition of paywalled O'Reilly book content (AUROC = 82\%), compared to OpenAI's earlier model GPT-3.5 Turbo. In contrast, GPT-3.5 Turbo shows greater relative recognition of publicly accessible O'Reilly book samples. GPT-4o Mini, as a much smaller model, shows no knowledge of public or non-public O'Reilly Media content when tested (AUROC 50\%). Testing multiple models, with the same cutoff date, helps us account for potential language shifts over time that might bias our findings. These results highlight the urgent need for increased corporate transparency regarding pre-training data sources as a means to develop formal licensing frameworks for AI content training
Paper Structure (13 sections, 4 figures, 5 tables)

This paper contains 13 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction: Identifying access violations
  2. Data and Methods
  3. Data: Public vs. non-public book data
  4. Methods: DE-COP and AUROC
  5. Findings: Did OpenAI train on copyrighted books?
  6. Robustness and limitations
  7. Discussion: Towards functional content AI marketplaces?
  8. Conclusion
  9. Appendix
  10. Additional Details About Our Dataset
  11. AUROC Results
  12. Prompts and Settings Used for Tested Models
  13. Prompts and Settings Used for Paraphrase model

Figures (4)

  • Figure 1: We split our sample of O'Reilly books by time period & accessibility.
  • Figure 2: AUROC Scores Showing Model Recognition of Pre-Training Data
  • Figure 3: DE-COP Guess Rate Improves: More capable models identify human text even when not trained on it.
  • Figure 4: AUROC score is highly dependent on the data scale and method it is measured with.