Singular Control of (Reflected) Brownian Motion: A Computational Method Suitable for Queueing Applications

TL;DR

This work tackles singular stochastic control problems in the positive orthant by approximating them with high-dimensional drift-control problems solvable via a computational method designed for dimensions up to at least 30. The core idea is to replace singular controls with drift controls bounded by $b$ and boundary-reflection components, yielding a drift-control problem whose solution—computed numerically—serves as a near-optimal proxy for the original problem as $b\to\infty$. The methodology is demonstrated across a suite of queueing-network examples in heavy traffic, including tandem queues, criss-cross networks, and many-queues-in-series, with diffusion (singular) policies derived from the EWF closely matching MD or optimal benchmarks and often outperforming naive policies. The approach scales to high dimension, provides practical implementable policies, and includes public code to facilitate replication and extension. Overall, the paper advances a scalable, simulation-based framework for designing near-optimal control policies in complex queueing systems via drift-control approximations and workload formulations.

Abstract

Motivated by applications in queueing theory, we consider a class of singular stochastic control problems whose state space is the d-dimensional positive orthant. The original problem is approximated by a drift control problem, to which we apply a recently developed computational method that is feasible for dimensions up to d=30 or more. To show that nearly optimal solutions are obtainable using this method, we present computational results for a variety of examples, including queueing network examples that have appeared previously in the literature.

Figures (13)

• Figure 1: A decomposable parallel-server queueing network.
• Figure 2: A network of tandem queues.
• Figure 3: Optimal policies for the singular control and MDP formulations.
• Figure 4: The criss-cross network.
• Figure 5: Graphical representation of the diffusion polices derived via solution of the approximating singular control problem.