Admission Control of Quasi-Reversible Queueing Systems: Optimization and Reinforcement Learning

Abstract

In this paper, we introduce a versatile scheme for optimizing the arrival rates of quasi-reversible queueing systems. We first propose an alternative definition of quasi-reversibility that encompasses reversibility and highlights the importance of the definition of customer classes. Then we introduce balanced arrival control policies, which generalize the notion of balanced arrival rates introduced in the context of Whittle networks, to the much broader class of quasi-reversible queueing systems. We prove that supplementing a quasi-reversible queueing system with a balanced arrival-control policy preserves the quasi-reversibility, and we specify the form of the stationary measures. We revisit two canonical examples of quasi-reversible queueing systems, Whittle networks and order-independent queues. Lastly, we focus on the problem of admission control and leverage our results in the frameworks of optimization and reinforcement learning.

Figures (15)

• Figure 1: An intuitive description of the balance condition in a toy queueing system of dimension $n = 2$ with macrostate space $\mathcal{X} = \{x \in \mathbb{N}^2\,;\, x_1 \le 4, x_2 \le 2\}$. For each $x \in \mathcal{X}$, $\Gamma(x)$ is the product of admission probabilities along an arbitrary increasing path going from the origin to $x$. The balance condition guarantees that $\Gamma(x)$ is well-defined in the sense that the product is independent of the path.
• Figure 2: A complete and consistent definition of the balanced admission probabilities in a system $n = 2$ classes and macrostate space $\mathcal{X} = \llbracket 0, 4 \rrbracket \times \llbracket 0, 2 \rrbracket$. The admission probabilities along the dashed arrows are fixed once we fix the admission probabilities along the solid edges.
• Figure 3: Comparison of the optimal policy and the best balanced policy in the toy example of \ref{['sec:cost_dimension']}. Blue edges represent admissions that are authorized (with probability 1) by the policy. The value inside each state gives the logarithm in base 10 of the value of the stationary distribution in this state under the depicted policy.
• Figure 4: Percentage loss of the best balanced policy with respect to the optimal policy, as a function of the per-class arrival rates in the toy example of \ref{['sec:cost_dimension']}.
• Figure 5: Comparison of the optimal policy and the best balanced policy in the toy example of \ref{['sec:cost_circulation']}. Blue edges represent admissions that are authorized (with probability 1) by the policy. The value inside each state gives the logarithm in base 10 of the value of the stationary distribution in this state under the depicted policy. We see that state $x = (5, 0)$ is more likely under the optimal policy ($\log \Pi^{\gamma^*}(5, 0) = -5.01$) than under the best balanced policy ($\log \Pi^{\delta \Gamma^*}(5, 0) = -5.05$).

Theorems & Definitions (19)

• Definition 1
• Definition 2: Balanced policy
• Theorem 3
• Corollary 4
• Lemma 5
• Proposition 6
• Example 7
• Lemma 8
• Definition 9
• Proposition 10