Table of Contents
Fetching ...

Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents

Shamse Tasnim Cynthia, Joy Krishan Das, Banani Roy

TL;DR

The paper investigates how core and peripheral developers interact with autonomous coding agents in open-source software. It analyzes 9,427 agentic PRs across Claude Code, Cursor, Copilot, and OpenAI Codex using a PR-based developer experience metric, a GPT-4 labeled 10-category taxonomy for PR purposes, and manual IRR-validated coding of review and modification patterns. Key findings show that a subset of peripheral contributors drives many agentic PRs, while core developers achieve higher acceptance on main branches, engage more in reviews, and enforce CI checks more consistently; most agentic PRs are accepted without developer modification, with refactoring being common when changes occur. The study offers implications for tooling, governance, and training to foster trustworthy human–AI collaboration and informs policies to mitigate socio-technical inequities in AI-assisted development. Overall, it provides empirical evidence on how developer experience shapes delegation, review, modification, and verification in agent-assisted software engineering, guiding practical integration of coding agents into real-world workflows.

Abstract

Autonomous AI agents are transforming software development and redefining how developers collaborate with AI. Prior research shows that the adoption and use of AI-powered tools differ between core and peripheral developers. However, it remains unclear how this dynamic unfolds in the emerging era of autonomous coding agents. In this paper, we present the first empirical study of 9,427 agentic PRs, examining how core and peripheral developers use, review, modify, and verify agent-generated contributions prior to acceptance. Through a mix of qualitative and quantitative analysis, we make four key contributions. First, a subset of peripheral developers use agents more often, delegating tasks evenly across bug fixing, feature addition, documentation, and testing. In contrast, core developers focus more on documentation and testing, yet their agentic PRs are frequently merged into the main/master branch. Second, core developers engage slightly more in review discussions than peripheral developers, and both groups focus on evolvability issues. Third, agentic PRs are less likely to be modified, but when they are, both groups commonly perform refactoring. Finally, peripheral developers are more likely to merge without running CI checks, whereas core developers more consistently require passing verification before acceptance. Our analysis offers a comprehensive view of how developer experience shapes integration offer insights for both peripheral and core developers on how to effectively collaborate with coding agents.

Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents

TL;DR

The paper investigates how core and peripheral developers interact with autonomous coding agents in open-source software. It analyzes 9,427 agentic PRs across Claude Code, Cursor, Copilot, and OpenAI Codex using a PR-based developer experience metric, a GPT-4 labeled 10-category taxonomy for PR purposes, and manual IRR-validated coding of review and modification patterns. Key findings show that a subset of peripheral contributors drives many agentic PRs, while core developers achieve higher acceptance on main branches, engage more in reviews, and enforce CI checks more consistently; most agentic PRs are accepted without developer modification, with refactoring being common when changes occur. The study offers implications for tooling, governance, and training to foster trustworthy human–AI collaboration and informs policies to mitigate socio-technical inequities in AI-assisted development. Overall, it provides empirical evidence on how developer experience shapes delegation, review, modification, and verification in agent-assisted software engineering, guiding practical integration of coding agents into real-world workflows.

Abstract

Autonomous AI agents are transforming software development and redefining how developers collaborate with AI. Prior research shows that the adoption and use of AI-powered tools differ between core and peripheral developers. However, it remains unclear how this dynamic unfolds in the emerging era of autonomous coding agents. In this paper, we present the first empirical study of 9,427 agentic PRs, examining how core and peripheral developers use, review, modify, and verify agent-generated contributions prior to acceptance. Through a mix of qualitative and quantitative analysis, we make four key contributions. First, a subset of peripheral developers use agents more often, delegating tasks evenly across bug fixing, feature addition, documentation, and testing. In contrast, core developers focus more on documentation and testing, yet their agentic PRs are frequently merged into the main/master branch. Second, core developers engage slightly more in review discussions than peripheral developers, and both groups focus on evolvability issues. Third, agentic PRs are less likely to be modified, but when they are, both groups commonly perform refactoring. Finally, peripheral developers are more likely to merge without running CI checks, whereas core developers more consistently require passing verification before acceptance. Our analysis offers a comprehensive view of how developer experience shapes integration offer insights for both peripheral and core developers on how to effectively collaborate with coding agents.
Paper Structure (17 sections, 7 figures, 7 tables)

This paper contains 17 sections, 7 figures, 7 tables.

Figures (7)

  • Figure 1: Overview of the methodology to investigate core and peripheral developers' use of coding agents
  • Figure 2: Distribution of resolved PRs across core and peripheral developers ($\bullet$ represents mean here)
  • Figure 3: Purposes of agentic PRs across core and peripheral developers
  • Figure 4: Distribution of reviews comments across peripheral and core developers ($\bullet$ represents mean here)
  • Figure 5: Prevalence of code review comments across developer experience groups
  • ...and 2 more figures