Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CI/CD Configuration Practices in Open-Source Android Apps: An Empirical Study

Taher A. Ghaleb, Osamah Abduljalil, Safwat Hassan

TL;DR

This empirical study analyzes CI/CD configurations in 2,557 open-source Android apps across four services (GitHub Actions, Travis CI, CircleCI, GitLab CI/CD) to understand complexity, evolution, and maintenance themes. The authors extract 24,976 directive/value pairs from 4,295 YAML files, categorize directives into five CI/CD phases, and validate themes with both manual coding and LDA topic modeling. Key findings show non-standardized configurations with a strong emphasis on Setup&Initialization, limited deployment (9%), and Gradle-based unit testing being prevalent (78%), while maintenance is bi-monthly on average and correlates with issue activity and project maturity. The work highlights a need for standardized Android-specific CI/CD practices and AI-assisted automation to streamline testing and deployment, offering practical directions for researchers and CI/CD tool developers.

Abstract

Continuous Integration and Continuous Delivery (CI/CD) is a well-established practice that automatically builds, tests, packages, and deploys software systems. To adopt CI/CD, software developers need to configure their projects using dedicated YML configuration files. Mobile apps have distinct characteristics with respect to CI/CD practices, such as testing on various emulators and deploying to app stores. However, little is known about the challenges and added value of adopting CI/CD in mobile apps and how developers maintain such a practice. In this paper, we conduct an empirical study on CI/CD practices in 2,557 Android apps adopting four popular CI/CD services, namely GitHub Actions, Travis CI, CircleCI, and GitLab CI/CD. We also compare our findings with those reported in prior research on general CI/CD practices to situate them within broader trends. We observe a lack of commonality and standardization across CI/CD services and Android apps, leading to complex YML configurations and associated maintenance efforts. We also observe that CI/CD configurations focus primarily on the build setup, with around half of the projects performing standard testing and only 9% incorporating deployment. In addition, we find that CI/CD configurations are changed bi-monthly on average, with frequent maintenance correlating with active issue tracking, project size/age, and community engagement. Our qualitative analysis of commits uncovered 11 themes in CI/CD maintenance activities, with over a third of the changes focusing on improving workflows and fixing build issues, whereas another third involves updating the build environment, tools, and dependencies. Our study emphasizes the necessity for automation and AI-powered tools to improve CI/CD processes for mobile apps and advocates creating adaptable open-source tools to efficiently manage resources, especially in testing and deployment.

CI/CD Configuration Practices in Open-Source Android Apps: An Empirical Study

TL;DR

This empirical study analyzes CI/CD configurations in 2,557 open-source Android apps across four services (GitHub Actions, Travis CI, CircleCI, GitLab CI/CD) to understand complexity, evolution, and maintenance themes. The authors extract 24,976 directive/value pairs from 4,295 YAML files, categorize directives into five CI/CD phases, and validate themes with both manual coding and LDA topic modeling. Key findings show non-standardized configurations with a strong emphasis on Setup&Initialization, limited deployment (9%), and Gradle-based unit testing being prevalent (78%), while maintenance is bi-monthly on average and correlates with issue activity and project maturity. The work highlights a need for standardized Android-specific CI/CD practices and AI-assisted automation to streamline testing and deployment, offering practical directions for researchers and CI/CD tool developers.

Abstract

Continuous Integration and Continuous Delivery (CI/CD) is a well-established practice that automatically builds, tests, packages, and deploys software systems. To adopt CI/CD, software developers need to configure their projects using dedicated YML configuration files. Mobile apps have distinct characteristics with respect to CI/CD practices, such as testing on various emulators and deploying to app stores. However, little is known about the challenges and added value of adopting CI/CD in mobile apps and how developers maintain such a practice. In this paper, we conduct an empirical study on CI/CD practices in 2,557 Android apps adopting four popular CI/CD services, namely GitHub Actions, Travis CI, CircleCI, and GitLab CI/CD. We also compare our findings with those reported in prior research on general CI/CD practices to situate them within broader trends. We observe a lack of commonality and standardization across CI/CD services and Android apps, leading to complex YML configurations and associated maintenance efforts. We also observe that CI/CD configurations focus primarily on the build setup, with around half of the projects performing standard testing and only 9% incorporating deployment. In addition, we find that CI/CD configurations are changed bi-monthly on average, with frequent maintenance correlating with active issue tracking, project size/age, and community engagement. Our qualitative analysis of commits uncovered 11 themes in CI/CD maintenance activities, with over a third of the changes focusing on improving workflows and fixing build issues, whereas another third involves updating the build environment, tools, and dependencies. Our study emphasizes the necessity for automation and AI-powered tools to improve CI/CD processes for mobile apps and advocates creating adaptable open-source tools to efficiently manage resources, especially in testing and deployment.

Paper Structure

This paper contains 44 sections, 1 equation, 6 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Background
  3. CI/CD Process
  4. CI/CD Configuration in Different Services
  5. GitHub Actions
  6. Travis CI
  7. CircleCI
  8. GitLab CI/CD
  9. Study setup
  10. Data Collection
  11. Collecting Android projects from F-Droid
  12. Collecting Android projects from GitHub
  13. Merging all projects from the two sources
  14. Identifying projects that adopt CI/CD
  15. Removing toy projects
  16. ...and 29 more sections

Figures (6)

  • Figure 1: General overview of the CI/CD process.
  • Figure 2: An overview of our data collection process
  • Figure 3: Boxplots showing log-scaled distributions of repository-level metrics the $2,557$ studied projects, with medians annotated and highlighted in orange color
  • Figure 4: Boxplots showing the total number of directives across CI/CD services, alongside the respective ratios of directives for each category. Icons:GitHub Actions • Travis CI • CircleCI • GitLab CI/CD
  • Figure 5: Boxplots showing the number of CI/CD configuration changes (commits), overall and across CI/CD services
  • ...and 1 more figures