Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Cost of Downgrading Build Systems: A Case Study of Kubernetes

Gareema Ranjan, Mahmoud Alfadel, Gengyi Sun, Shane McIntosh

TL;DR

The paper analyzes the performance and cost consequences of downgrading from Bazel to Go Build using Kubernetes as a large-scale case study. By replaying full and incremental builds across multiple parallelism levels and measuring duration, memory, CPU, and CI cost, the study shows Bazel is faster but memory- and CPU-intensive, while Go Build is more memory-efficient but less scalable at scale. The authors extend the analysis through replication on four smaller projects and a time-window expansion, finding that memory overhead for Bazel remains consistent and speed advantages vary with project size. The work provides actionable guidance for tool choice based on project scale, hardware resources, and CI budgeting, and highlights the need for build systems that balance performance with maintainability. Overall, artifact-based tools offer speed advantages at scale but require careful consideration of resource and cost implications when downgrading to language-specific builds.

Abstract

Since developers invoke the build system frequently, its performance can impact productivity. Modern artifact-based build tools accelerate builds, yet prior work shows that teams may abandon them for alternatives that are easier to maintain. While prior work shows why downgrades are performed, the implications of downgrades remain largely unexplored. In this paper, we describe a case study of the Kubernetes project, focusing on its downgrade from an artifact-based build tool (Bazel) to a language-specific solution (Go Build). We reproduce and analyze the full and incremental builds of change sets during the downgrade period. On the one hand, we find that Bazel builds are faster than Go Build, completing full builds in 23.06-38.66 up to 75.19 impose a larger memory footprint than Go Build of 81.42-351.07 respectively. Bazel builds also impose a greater CPU load at parallelism settings above eight for full builds and above one for incremental builds. We estimate that downgrading from Bazel can increase CI resource costs by up to 76 explore whether our observations generalize by replicating our Kubernetes study on four other projects that also downgraded from Bazel to older build tools. We observe that while build time penalties decrease, Bazel consistently consumes more memory. We conclude that abandoning artifact-based build tools, despite perceived maintainability benefits, tends to incur considerable performance costs for large projects. Our observations may help stakeholders to balance trade-offs in build tool adoption

The Cost of Downgrading Build Systems: A Case Study of Kubernetes

TL;DR

The paper analyzes the performance and cost consequences of downgrading from Bazel to Go Build using Kubernetes as a large-scale case study. By replaying full and incremental builds across multiple parallelism levels and measuring duration, memory, CPU, and CI cost, the study shows Bazel is faster but memory- and CPU-intensive, while Go Build is more memory-efficient but less scalable at scale. The authors extend the analysis through replication on four smaller projects and a time-window expansion, finding that memory overhead for Bazel remains consistent and speed advantages vary with project size. The work provides actionable guidance for tool choice based on project scale, hardware resources, and CI budgeting, and highlights the need for build systems that balance performance with maintainability. Overall, artifact-based tools offer speed advantages at scale but require careful consideration of resource and cost implications when downgrading to language-specific builds.

Abstract

Since developers invoke the build system frequently, its performance can impact productivity. Modern artifact-based build tools accelerate builds, yet prior work shows that teams may abandon them for alternatives that are easier to maintain. While prior work shows why downgrades are performed, the implications of downgrades remain largely unexplored. In this paper, we describe a case study of the Kubernetes project, focusing on its downgrade from an artifact-based build tool (Bazel) to a language-specific solution (Go Build). We reproduce and analyze the full and incremental builds of change sets during the downgrade period. On the one hand, we find that Bazel builds are faster than Go Build, completing full builds in 23.06-38.66 up to 75.19 impose a larger memory footprint than Go Build of 81.42-351.07 respectively. Bazel builds also impose a greater CPU load at parallelism settings above eight for full builds and above one for incremental builds. We estimate that downgrading from Bazel can increase CI resource costs by up to 76 explore whether our observations generalize by replicating our Kubernetes study on four other projects that also downgraded from Bazel to older build tools. We observe that while build time penalties decrease, Bazel consistently consumes more memory. We conclude that abandoning artifact-based build tools, despite perceived maintainability benefits, tends to incur considerable performance costs for large projects. Our observations may help stakeholders to balance trade-offs in build tool adoption
Paper Structure (15 sections, 5 figures, 3 tables)

This paper contains 15 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Study Design
  3. Primary Subject System
  4. Data and Environment Preparation (DEP)
  5. Replay Execution (RE)
  6. Primary Study Results
  7. Analytic Generalizability
  8. Replication on Another Time Window
  9. Replication on Other Projects
  10. Qualitative Comparison of Build Tools
  11. Threats to Validity
  12. Related Work
  13. Build Migration
  14. Build Performance
  15. Final Remarks and Lessons Learned

Figures (5)

  • Figure 1: Median full and incremental build durations for Kubernetes.
  • Figure 2: Median memory consumption (in GiB) for full builds and incremental builds for Kubernetes.
  • Figure 3: Distribution of median values of CPU usage (%) for full builds and incremental builds for Kubernetes.
  • Figure 4: Build cost analysis for Kubernetes using Bazel and Go Build.
  • Figure 5: Performance metrics for analytic generalizability analysis.