Utility maximizing load balancing policies

TL;DR

This work addresses load balancing in a large-scale service system with heterogeneous server pools endowed with occupancy-dependent concave utilities. It formulates a problem-structured upper bound on mean stationary utility and designs two policies, JLMU and SLTA, that are asymptotically optimal in the large-server-pool limit under exponential service times. The authors develop fluid-limit models, strong approximations, and limit theorems to establish convergence of occupancy profiles to an optimal target q_* (and thresholds to σ_*) and show that E[u(q_n)] converges to u(q_*). Simulation results corroborate the theoretical findings, illustrating near-optimal performance even for moderate system sizes. The results provide a principled benchmark for utility-based load balancing in heterogeneous, infinite-server settings and demonstrate practical threshold-learning mechanisms that adapt to unknown offered load.

Abstract

Consider a service system where incoming tasks are instantaneously dispatched to one out of many heterogeneous server pools. Associated with each server pool is a concave utility function which depends on the class of the server pool and its current occupancy. We derive an upper bound for the mean normalized aggregate utility in stationarity and introduce two load balancing policies that achieve this upper bound in a large-scale regime. Furthermore, the transient and stationary behavior of these asymptotically optimal load balancing policies is characterized on the scale of the number of server pools, in the same large-scale regime.

Figures (6)

• Figure 1: Schematic view of some of the related work. Most of the load balancing literature concerns systems of parallel and homogeneous single-server queues; this vast literature is surveyed in van2018scalable. Some recent papers study single-server dynamics in heterogeneous settings or infinite-server dynamics in homogeneous settings, whereas the present paper considers a heterogeneous system with infinite-server dynamics.
• Figure 2: Schematic representation of the marginal utilities. White rectangular slots and gray rectangles represent idle and busy servers, respectively. Each of the columns labeled with letters represents a server pool and the dashed lines enclose server pools of the same class. If the tasks sent to a given server pool are always placed in the first idle server from bottom to top, then the marginal utilities written on top of the idle servers indicate the increase in the aggregate utility of the system when the server receives a task.
• Figure 3: Distribution of the offered load across the various server pools under the optimal task assignment of \ref{['eq: definition of q_n^N']} for $m = 3$ and $\alpha_n = (1/2, 1/4, 1/4)$. On the left, $u_i(x) = x \log(r(i) / x)$ with $r = (5, 10, 15)$. On the right, $u_1(x) = x$, $u_2(x) = 2x - x^2/20$, $u_3(x) = 3x/2$ if $x < 20$ and $u_3(x) = 30$ if $x \geq 20$.
• Figure 4: Schematic representation of the thresholds and tokens used by SLTA for $(i_{\boldsymbol{r}_n}, j_{\boldsymbol{r}_n}) = (2, 2)$. The thresholds are indicated by thick horizontal lines that cross the server pools, and the tokens are represented using squares and circles underneath the server pools; a square corresponds to a green token and a circle corresponds to a yellow token. Assuming that the rectangular slots within a server pool are always filled from bottom to top, the slots marked with an $*$ provide a marginal utility of $\Delta(i_{\boldsymbol{r}_n}, j_{\boldsymbol{r}_n} - 1)$. The symbols $\leq$ and $\geq$ indicate how the marginal utility of the other slots compares to the latter value.
• Figure 5: Schematic view of the proofs of (a) and (b) of Theorem \ref{['the: asymptotic optimality']}, on the left and right, respectively.

Theorems & Definitions (64)

• Theorem 1
• Theorem 2
• Definition 1
• Theorem 3
• Theorem 4
• Remark 1
• Theorem 5
• Theorem 6
• Lemma 1
• proof