Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Source

Haoyu Gao, Peerachai Banyongrakkul, Hao Guan, Mansooreh Zahedi, Christoph Treude

TL;DR

This paper investigates human-AI teaming in open source by expanding the AIDev dataset to include contributor code ownership and a human PR baseline, then analyzes adoption trends, governance artefacts, and review practices across AI-assisted and human-created PRs. It finds that $67.5\%$ of AI-coauthored PRs come from contributors with no prior ownership, and $86.9\%$ of projects lack AI usage guidelines, while AI-assisted PRs are merged faster with minimal feedback—especially for non-owners—compared to human PRs, which attract more scrutiny. The results highlight governance gaps and suggest the need for explicit AI usage guidelines and task-allocation strategies to balance efficiency with code quality and inclusive collaboration. The findings have practical implications for OSS maintainers and researchers aiming to design robust governance frameworks for AI-enabled software development.

Abstract

Large Language Models (LLMs) increasingly automate software engineering tasks. While recent studies highlight the accelerated adoption of ``AI as a teammate'' in Open Source Software (OSS), developer interaction patterns remain under-explored. In this work, we investigated project-level guidelines and developers' interactions with AI-assisted pull requests (PRs) by expanding the AIDev dataset to include finer-grained contributor code ownership and a comparative baseline of human-created PRs. We found that over 67.5\% of AI-co-authored PRs originate from contributors without prior code ownership. Despite this, the majority of repositories lack guidelines for AI-coding agent usage. Notably, we observed a distinct interaction pattern: AI-co-authored PRs are merged significantly faster with minimal feedback. In contrast to human-created PRs where non-owner developers receive the most feedback, AI-co-authored PRs from non-owners receive the least, with approximately 80\% merged without any explicit review. Finally, we discuss implications for developers and researchers.

On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Source

TL;DR

This paper investigates human-AI teaming in open source by expanding the AIDev dataset to include contributor code ownership and a human PR baseline, then analyzes adoption trends, governance artefacts, and review practices across AI-assisted and human-created PRs. It finds that of AI-coauthored PRs come from contributors with no prior ownership, and of projects lack AI usage guidelines, while AI-assisted PRs are merged faster with minimal feedback—especially for non-owners—compared to human PRs, which attract more scrutiny. The results highlight governance gaps and suggest the need for explicit AI usage guidelines and task-allocation strategies to balance efficiency with code quality and inclusive collaboration. The findings have practical implications for OSS maintainers and researchers aiming to design robust governance frameworks for AI-enabled software development.

Abstract

Large Language Models (LLMs) increasingly automate software engineering tasks. While recent studies highlight the accelerated adoption of ``AI as a teammate'' in Open Source Software (OSS), developer interaction patterns remain under-explored. In this work, we investigated project-level guidelines and developers' interactions with AI-assisted pull requests (PRs) by expanding the AIDev dataset to include finer-grained contributor code ownership and a comparative baseline of human-created PRs. We found that over 67.5\% of AI-co-authored PRs originate from contributors without prior code ownership. Despite this, the majority of repositories lack guidelines for AI-coding agent usage. Notably, we observed a distinct interaction pattern: AI-co-authored PRs are merged significantly faster with minimal feedback. In contrast to human-created PRs where non-owner developers receive the most feedback, AI-co-authored PRs from non-owners receive the least, with approximately 80\% merged without any explicit review. Finally, we discuss implications for developers and researchers.
Paper Structure (9 sections, 2 figures, 2 tables)

This paper contains 9 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Study Design and Methodology
  4. Data Collection
  5. Data Analysis
  6. Results and Discussion
  7. Threats to Validity
  8. Ethical Implications
  9. Conclusion

Figures (2)

  • Figure 1: Average Human vs AI PRs across the active months
  • Figure 2: Contributors' Onwership Distribution and Contribution Volume for Human and AI PRs