Asynchronous Load Balancing and Auto-scaling: Mean-Field Limit and Optimal Design

TL;DR

This work develops ALBA, an asynchronous, decentralized framework that jointly models load balancing and auto-scaling in serverless-like environments. By formulating a Markov model and its mean-field fluid limit, the authors derive a general optimality condition for scaling rules that yields vanishing delay and relative energy waste in the large-system limit, showing that capacity should be increased when $ ext{mean demand} > eta x_{0,1} + x_{1,2}$. They propose Rate-Idle (and related mappings) and prove fluid-optimal convergence under JIQ, with numerical results demonstrating superior delay performance over synchronous schemes while achieving comparable energy usage. The paper also characterizes fixed points, demonstrates possible convergence to suboptimal points under certain rules, and provides an optimization framework to trade off delay and energy in practical deployments. Overall, the results offer a tractable design paradigm for scalable, energy-conscious, asynchronous serverless orchestration with theoretical performance guarantees and actionable policies.

Abstract

We develop a Markovian framework for load balancing that combines classical algorithms such as Power-of-$d$ with auto-scaling mechanisms that allow the net service capacity to scale up or down in response to the current load on the same timescale as job dynamics. Our framework is inspired by serverless platforms, such as Knative, where servers are software functions that can be flexibly instantiated in milliseconds according to scaling rules defined by the users of the serverless platform. The main question is how to design such scaling rules to minimize user-perceived delay performance while ensuring low energy consumption. For the first time, we investigate this problem when the auto-scaling and load balancing processes operate asynchronously (or proactively), as in Knative. In contrast to the synchronous (or reactive) paradigm, asynchronism brings the advantage that jobs do not necessarily need to wait any time a scale-up decision is taken. In our main result, we find a general condition on the structure of scaling rules able to drive mean-field dynamics to delay and relative energy optimality, i.e., a situation where both the user-perceived delay and the relative energy waste induced by idle servers vanish in the limit where the network demand grows to infinity in proportion to the nominal service capacity. The identified condition suggests to scale up the current net capacity if and only if the mean demand exceeds the rate at which servers become idle and active. Finally, we propose a family of scaling rules that satisfy our optimality condition. Numerical simulations demonstrate that these rules provide better delay performance than existing synchronous auto-scaling schemes while inducing almost the same power consumption.

Figures (3)

• Figure 1: Numerical convergence of the stochastic model $X^N(t)$ (continuous lines), $N=10^3$, to the fluid model $x(t)$ (dashed lines) when combining Power-of-2 and Blind-$\theta$.
• Figure 2: Ratio $\mathcal{R}^{(N)}$ of the transient probability of waiting induced by the proposed asynchronous scheme (Rate-Idle+JIQ) and the synchronous approach in elasticSIG, respectively. The initial condition is the global attractor $x^\star$ (defined in Section \ref{['sec:optimal_design']}, see Theorem \ref{['th3']}), which corresponds to delay and relative energy optimality.
• Figure 3: Transient behavior of the queue lengths ($y$-axis on the left) and scaling probabilities ($y$-axis on the right) by varying $\eta$, see \ref{['gx_eta']}, for both the fluid ($x(t)$) and stochastic ($X^N(t)$) models with $N=1000$.

Theorems & Definitions (33)

• Definition 1
• Definition 2
• Theorem 1
• proof
• Theorem 2
• proof
• Remark 1: Multiple Fixed Points
• Remark 2: Fluid Optimality
• Remark 3
• Theorem 3: Optimal Design