Self-Admitted GenAI Usage in Open-Source Software

TL;DR

This study introduces self-admitted GenAI usage as an empirical lens to study open source software development in the era of AI assisted coding. By mining over 250,000 GitHub repositories, applying qualitative coding to 1,292 self-admitted mentions across 156 projects, and analyzing policies plus a developer survey, the authors derive a taxonomy of 32 GenAI assisted tasks, 10 content types, and 11 usage purposes. They show that projects actively regulate GenAI usage through varied governance, and their longitudinal churn analysis across 149 repositories finds no general increase in code churn after adoption, though generation tasks can be more prone to rework in certain contexts. The work underscores the need for project level transparency, provenance, and tailored governance, and provides a rich data set and methods for future replication and large scale analysis of AI assisted software development.

Abstract

The widespread adoption of generative AI (GenAI) tools such as GitHub Copilot and ChatGPT is transforming software development. Since generated source code is virtually impossible to distinguish from manually written code, their real-world usage and impact on open-source software development remain poorly understood. In this paper, we introduce the concept of self-admitted GenAI usage, that is, developers explicitly referring to the use of GenAI tools for content creation in software artifacts. Using this concept as a lens to study how GenAI tools are integrated into open-source software projects, we analyze a curated sample of more than 250,000 GitHub repositories, identifying 1,292 such self-admissions across 156 repositories in commit messages, code comments, and project documentation. Using a mixed methods approach, we derive a taxonomy of 32 tasks, 10 content types, and 11 purposes associated with GenAI usage based on 1,292 qualitatively coded mentions. We then analyze 13 documents with policies and usage guidelines for GenAI tools and conduct a developer survey to uncover the ethical, legal, and practical concerns behind them. Our findings reveal that developers actively manage how GenAI is used in their projects, highlighting the need for project-level transparency, attribution, and quality control practices in the new era of AI-assisted software development. Finally, we examine the longitudinal impact of GenAI adoption on code churn in 151 repositories with self-admitted GenAI usage and find no general increase, contradicting popular narratives on the impact of GenAI on software development.

Figures (2)

• Figure 1: Overview of the data collection process used to answer our three research questions, from the selection (1) and filtering (2) of GitHub repositories to the extraction of GenAI mentions (3) and the identification of self-admitted GenAI usage (4) in these repositories.
• Figure 2: Selected examples for RDD analysis of file churn (RQ3).