Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans

Yangtian Zi, Zixuan Wu, Aleksander Boruch-Gruszecki, Jonathan Bell, Arjun Guha

Abstract

Fine-tuning large language models for code editing has typically relied on mining commits and pull requests. The working hypothesis has been that commit messages describe human intent in natural language, and patches to code describe the changes that implement that intent. However, much of the previously collected data is noisy: commit messages are terse, human-written commits commingle several unrelated edits, and many commits come from simple, rule-based bots. The recent adoption of software engineering agents changes this landscape. Code changes \emph{co-authored} by humans and agents are often accompanied by substantially more explicit natural-language descriptions of intent and rationale. Moreover, when these changes land in public repositories, they are implicitly filtered by humans: maintainers discard low-quality commits to their projects. We present AgentPack, a corpus of 1.8M code edits co-authored by Claude Code, OpenAI Codex, and Cursor Agent across public GitHub projects up to early October 2025. We describe the identification and curation pipeline, quantify adoption trends of these agents, and analyze the structural properties of the edits. Finally, we show that models fine-tuned on AgentPack can outperform models trained on prior human-only commit corpora, highlighting the potential of using public data from software engineering agents to train future code-editing models.

AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans

Abstract

Fine-tuning large language models for code editing has typically relied on mining commits and pull requests. The working hypothesis has been that commit messages describe human intent in natural language, and patches to code describe the changes that implement that intent. However, much of the previously collected data is noisy: commit messages are terse, human-written commits commingle several unrelated edits, and many commits come from simple, rule-based bots. The recent adoption of software engineering agents changes this landscape. Code changes \emph{co-authored} by humans and agents are often accompanied by substantially more explicit natural-language descriptions of intent and rationale. Moreover, when these changes land in public repositories, they are implicitly filtered by humans: maintainers discard low-quality commits to their projects. We present AgentPack, a corpus of 1.8M code edits co-authored by Claude Code, OpenAI Codex, and Cursor Agent across public GitHub projects up to early October 2025. We describe the identification and curation pipeline, quantify adoption trends of these agents, and analyze the structural properties of the edits. Finally, we show that models fine-tuned on AgentPack can outperform models trained on prior human-only commit corpora, highlighting the potential of using public data from software engineering agents to train future code-editing models.

Paper Structure

This paper contains 31 sections, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Background and Related Work
  4. Datasets for Fine-Tuning Models on Coding Tasks
  5. Reinforcement Learning for Coding Tasks
  6. Training Models to Edit Code
  7. Benchmarking Models on Code Editing Tasks
  8. Analyzing Open Source Code at Scale
  9. Building AgentPack
  10. Exploring AgentPack
  11. The Code in AgentPack
  12. Summary statistics
  13. Comparison to other code editing datasets
  14. The programming languages in AgentPack
  15. The Types of Tasks in AgentPack
  16. ...and 16 more sections

Figures (6)

  • Figure 1: Total commits and new pull requests made by Cursor Agent, Claude Code, and Codex from launch to the indicated date.
  • Figure 2: Examples of three natural language descriptions authored by three different agents from AgentPack. In the captions above, we list the labels that we assigned to each of these descriptions for our classification.
  • Figure 3: Distribution of commit labels across three coding agents (Claude Code, Codex, and Cursor Agent), based on a random sample of 5000 commits from each agents. Each bar represents the frequency of commits associated with a given label. A single commit may carry multiple labels, and counts reflect total label occurrences rather than single commits.
  • Figure 4: Prompt templates used in our main fine-tuning evaluations: (a) HumanEvalFix and (b) CanItEdit.
  • Figure 5: CanItEdit 1-shot prompt template.
  • ...and 1 more figures