Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Possibility of Breaking Copyleft Licenses When Reusing Code Generated by ChatGPT

Gaia Colombo, Leonardo Mariani, Daniela Micucci, Oliviero Riganelli

TL;DR

This study investigates the risk that ChatGPT-recommended code mirrors copyleft-licensed implementations. Using a large Java-method dataset derived from copyleft repositories and more than 70,000 generated methods, the authors quantify similarity to known copyleft code with JPlag and fuzzy metrics. They show that while single requests mostly avoid copying, expanding context within prompts and including access methods dramatically increases the chance of reproducing copyleft code; higher temperature reduces this risk but may affect correctness. The results imply actionable guidance for practitioners: limit context, tune temperature, and remain vigilant about licensing when using AI-assisted code generation, with future work on detection mechanisms and broader model generalization.

Abstract

AI assistants can help developers by recommending code to be included in their implementations (e.g., suggesting the implementation of a method from its signature). Although useful, these recommendations may mirror copyleft code available in public repositories, exposing developers to the risk of reusing code that they are allowed to reuse only under certain constraints (e.g., a specific license for the derivative software). This paper presents a large-scale study about the frequency and magnitude of this phenomenon in ChatGPT. In particular, we generate more than 70,000 method implementations using a range of configurations and prompts, revealing that a larger context increases the likelihood of reproducing copyleft code, but higher temperature settings can mitigate this issue.

On the Possibility of Breaking Copyleft Licenses When Reusing Code Generated by ChatGPT

TL;DR

This study investigates the risk that ChatGPT-recommended code mirrors copyleft-licensed implementations. Using a large Java-method dataset derived from copyleft repositories and more than 70,000 generated methods, the authors quantify similarity to known copyleft code with JPlag and fuzzy metrics. They show that while single requests mostly avoid copying, expanding context within prompts and including access methods dramatically increases the chance of reproducing copyleft code; higher temperature reduces this risk but may affect correctness. The results imply actionable guidance for practitioners: limit context, tune temperature, and remain vigilant about licensing when using AI-assisted code generation, with future work on detection mechanisms and broader model generalization.

Abstract

AI assistants can help developers by recommending code to be included in their implementations (e.g., suggesting the implementation of a method from its signature). Although useful, these recommendations may mirror copyleft code available in public repositories, exposing developers to the risk of reusing code that they are allowed to reuse only under certain constraints (e.g., a specific license for the derivative software). This paper presents a large-scale study about the frequency and magnitude of this phenomenon in ChatGPT. In particular, we generate more than 70,000 method implementations using a range of configurations and prompts, revealing that a larger context increases the likelihood of reproducing copyleft code, but higher temperature settings can mitigate this issue.

Paper Structure

This paper contains 15 sections, 16 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Code Generation with LLM
  4. Plagiarism in AI-generated Code
  5. Methodology
  6. Dataset Construction
  7. Plagiarism Assessment
  8. Results
  9. RQ1: Can ChatGPT return code protected by copyleft licenses?
  10. RQ2: Does the context provided in the request influence the likelihood of returning code protected by copyleft licenses?
  11. RQ3: Does providing a context with only access methods affect the likelihood of obtaining code protected by copyleft licenses?
  12. RQ4: Does adjusting the temperature parameter alter the likelihood of obtaining code protected by copyleft licenses?
  13. RQ5: Is ChatGPT aware of using protected code and can it avoid plagiarism when explicitly requested?
  14. Threats to Validity
  15. Conclusions

Figures (16)

  • Figure 1: Distribution of similarity scores for mean similarity.
  • Figure 2: Distribution of similarity scores for max similarity.
  • Figure 3: Difference between JavaDoc comment and whole class in the case of mean similarity.
  • Figure 4: Difference between JavaDoc comment and whole class in the case of max similarity.
  • Figure 5: Similarity Scores for JavaDoc comment and whole class contexts in the case of mean similarity.
  • ...and 11 more figures