Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GitHub Actions: The Impact on the Pull Request Process

Mairieli Wessel, Joseph Vargovich, Marco A. Gerosa, Christoph Treude

TL;DR

This study investigates how OSS projects adopt and use GitHub Actions and how adoption affects pull request dynamics. Employing a mixed-methods design, it analyzes 5,000 top-starred repositories to identify adoption patterns, categorizes nearly 1,000 Actions, and qualitatively codes discussions, complemented by a time-series regression discontinuity analysis on 662 active projects. The results show that about 30% of studied projects adopt Actions, with adoption associated with fewer merged PRs and more rejected PRs, longer time to close, and shifts in discussion intensity; effects vary by Action category, notably Code Quality reducing rejections. The work provides practical guidance for practitioners and contributes a nuanced empirical view of automation’s socio-technical impact on collaboration and contribution dynamics in OSS.

Abstract

Software projects frequently use automation tools to perform repetitive activities in the distributed software development process. Recently, GitHub introduced GitHub Actions, a feature providing automated workflows for software projects. Understanding and anticipating the effects of adopting such technology is important for planning and management. Our research investigates how projects use GitHub Actions, what the developers discuss about them, and how project activity indicators change after their adoption. Our results indicate that 1,489 out of 5,000 most popular repositories (almost 30% of our sample) adopt GitHub Actions and that developers frequently ask for help implementing them. Our findings also suggest that the adoption of GitHub Actions leads to more rejections of pull requests (PRs), more communication in accepted PRs and less communication in rejected PRs, fewer commits in accepted PRs and more commits in rejected PRs, and more time to accept a PR. We found similar results when segmenting our results by categories of GitHub Actions. We suggest practitioners consider these effects when adopting GitHub Actions on their projects.

GitHub Actions: The Impact on the Pull Request Process

TL;DR

This study investigates how OSS projects adopt and use GitHub Actions and how adoption affects pull request dynamics. Employing a mixed-methods design, it analyzes 5,000 top-starred repositories to identify adoption patterns, categorizes nearly 1,000 Actions, and qualitatively codes discussions, complemented by a time-series regression discontinuity analysis on 662 active projects. The results show that about 30% of studied projects adopt Actions, with adoption associated with fewer merged PRs and more rejected PRs, longer time to close, and shifts in discussion intensity; effects vary by Action category, notably Code Quality reducing rejections. The work provides practical guidance for practitioners and contributes a nuanced empirical view of automation’s socio-technical impact on collaboration and contribution dynamics in OSS.

Abstract

Software projects frequently use automation tools to perform repetitive activities in the distributed software development process. Recently, GitHub introduced GitHub Actions, a feature providing automated workflows for software projects. Understanding and anticipating the effects of adopting such technology is important for planning and management. Our research investigates how projects use GitHub Actions, what the developers discuss about them, and how project activity indicators change after their adoption. Our results indicate that 1,489 out of 5,000 most popular repositories (almost 30% of our sample) adopt GitHub Actions and that developers frequently ask for help implementing them. Our findings also suggest that the adoption of GitHub Actions leads to more rejections of pull requests (PRs), more communication in accepted PRs and less communication in rejected PRs, fewer commits in accepted PRs and more commits in rejected PRs, and more time to accept a PR. We found similar results when segmenting our results by categories of GitHub Actions. We suggest practitioners consider these effects when adopting GitHub Actions on their projects.
Paper Structure (21 sections, 1 equation, 3 figures, 16 tables)

This paper contains 21 sections, 1 equation, 3 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Workflow Automation with GitHub Actions
  3. Related Work
  4. Development Bots
  5. Continuous Integration and Continuous Delivery
  6. GitHub Actions
  7. Research Design
  8. Selecting Projects
  9. Analyzing the use of GitHub Actions
  10. Categorizing GitHub Actions Discussions
  11. Time series analysis
  12. Aggregating project variables
  13. Statistical Approach
  14. Results
  15. How do OSS projects use GitHub Actions? (RQ1)
  16. ...and 6 more sections

Figures (3)

  • Figure 1: GitHub workflow automation with GitHub Actions (adapted from GitHub).
  • Figure 2: Number of Actions per repository (log scale)
  • Figure :