Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When AI Teammates Meet Code Review: Collaboration Signals Shaping the Integration of Agent-Authored Pull Requests

Costain Nachuma, Minhaz Zibran

TL;DR

A large empirical study of agent-authored pull requests using the public AIDev dataset, examining integration outcomes, resolution speed, and review-time collaboration signals finds that reviewer engagement has the strongest correlation with successful integration.

Abstract

Autonomous coding agents increasingly contribute to software development by submitting pull requests on GitHub; yet, little is known about how these contributions integrate into human-driven review workflows. We present a large empirical study of agent-authored pull requests using the public AIDev dataset, examining integration outcomes, resolution speed, and review-time collaboration signals. Using logistic regression with repository-clustered standard errors, we find that reviewer engagement has the strongest correlation with successful integration, whereas larger change sizes and coordination-disrupting actions, such as force pushes, are associated with a lower likelihood of merging. In contrast, iteration intensity alone provides limited explanatory power once collaboration signals are considered. A qualitative analysis further shows that successful integration occurs when agents engage in actionable review loops that converge toward reviewer expectations. Overall, our results highlight that the effective integration of agent-authored pull requests depends not only on code quality but also on alignment with established review and coordination practices.

When AI Teammates Meet Code Review: Collaboration Signals Shaping the Integration of Agent-Authored Pull Requests

TL;DR

A large empirical study of agent-authored pull requests using the public AIDev dataset, examining integration outcomes, resolution speed, and review-time collaboration signals finds that reviewer engagement has the strongest correlation with successful integration.

Abstract

Autonomous coding agents increasingly contribute to software development by submitting pull requests on GitHub; yet, little is known about how these contributions integrate into human-driven review workflows. We present a large empirical study of agent-authored pull requests using the public AIDev dataset, examining integration outcomes, resolution speed, and review-time collaboration signals. Using logistic regression with repository-clustered standard errors, we find that reviewer engagement has the strongest correlation with successful integration, whereas larger change sizes and coordination-disrupting actions, such as force pushes, are associated with a lower likelihood of merging. In contrast, iteration intensity alone provides limited explanatory power once collaboration signals are considered. A qualitative analysis further shows that successful integration occurs when agents engage in actionable review loops that converge toward reviewer expectations. Overall, our results highlight that the effective integration of agent-authored pull requests depends not only on code quality but also on alignment with established review and coordination practices.
Paper Structure (29 sections, 1 equation, 2 figures, 1 table)

This paper contains 29 sections, 1 equation, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Dataset and Operational Definitions
  3. Integration and Resolution (RQ1)
  4. Motivation
  5. Approach
  6. Exploratory note on reverts.
  7. Results
  8. Outcome rates vary sharply by agent.
  9. Decision latency differs even more than merge rates.
  10. Summary
  11. Collaboration Signals (RQ2)
  12. Motivation
  13. Approach
  14. Results
  15. Reviewer engagement dominates integration outcomes.
  16. ...and 14 more sections

Figures (2)

  • Figure 1: Outcomes of agent-authored pull requests
  • Figure 2: Forest plot of logistic regression predictors