Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Queueing Theory for Large-Scale CI/CD Pipelines Optimization

Grégory Bournassenko

TL;DR

This paper addresses the optimization of large-scale CI/CD pipelines under shared infrastructure using queueing-theory methods. It starts from the classic $M/M/c$ model and extends to $M/G/c$ and $G/G/c$ to capture variability in service times and arrivals, integrating load forecasting and dynamic, cost-aware scaling. Key contributions include SMA, exponential smoothing, and ML-based demand forecasting; multi-objective cost functions with SLA constraints; task scheduling heuristics (FCFS, SPT, EDF); bottleneck analysis via Little's Law; and a case study validating the approach, plus extensions to heterogeneous runners and advanced queueing models such as $M/G/1$ and $G/G/1$. The framework provides practical guidance for achieving higher throughput and lower waiting in enterprise CI/CD systems through adaptive resource management and prioritization policies.

Abstract

Continuous Integration and Continuous Deployment (CI/CD) pipelines are central to modern software development. In large organizations, the high volume of builds and tests creates bottlenecks, especially under shared infrastructure. This article proposes a modeling framework based on queueing theory to optimize large-scale CI/CD workflows. We formalize the system using classical $M/M/c$ queueing models and discuss strategies to minimize delays and infrastructure costs. Our approach integrates theoretical results with practical techniques, including dynamic scaling and prioritization of CI/CD tasks.

On Queueing Theory for Large-Scale CI/CD Pipelines Optimization

TL;DR

This paper addresses the optimization of large-scale CI/CD pipelines under shared infrastructure using queueing-theory methods. It starts from the classic model and extends to and to capture variability in service times and arrivals, integrating load forecasting and dynamic, cost-aware scaling. Key contributions include SMA, exponential smoothing, and ML-based demand forecasting; multi-objective cost functions with SLA constraints; task scheduling heuristics (FCFS, SPT, EDF); bottleneck analysis via Little's Law; and a case study validating the approach, plus extensions to heterogeneous runners and advanced queueing models such as and . The framework provides practical guidance for achieving higher throughput and lower waiting in enterprise CI/CD systems through adaptive resource management and prioritization policies.

Abstract

Continuous Integration and Continuous Deployment (CI/CD) pipelines are central to modern software development. In large organizations, the high volume of builds and tests creates bottlenecks, especially under shared infrastructure. This article proposes a modeling framework based on queueing theory to optimize large-scale CI/CD workflows. We formalize the system using classical queueing models and discuss strategies to minimize delays and infrastructure costs. Our approach integrates theoretical results with practical techniques, including dynamic scaling and prioritization of CI/CD tasks.

Paper Structure

This paper contains 47 sections, 22 equations.

Table of Contents

  1. Introduction
  2. Extended Modeling
  3. The $M/G/c$ Queue
  4. The $G/G/c$ Queue
  5. Load Forecasting
  6. Simple Moving Average Forecasting
  7. Exponential Smoothing
  8. Machine Learning Approaches
  9. Advanced Cost Optimization
  10. Multi-Objective Cost Functions
  11. Service Level Agreements (SLAs)
  12. Task Scheduling Heuristics
  13. First-Come-First-Served (FCFS)
  14. Shortest Processing Time (SPT)
  15. Earliest Deadline First (EDF)
  16. ...and 32 more sections