Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers

Haozhe Chen, Ang Li, Ethan Che, Tianyi Peng, Jing Dong, Hongseok Namkoong

TL;DR

An open-sourced queueing simulation framework, QGym, that benchmark queueing policies across realistic problem instances, and makes it easy to compare multiple policies, including both model-free RL methods and classical queuing policies.

Abstract

Queuing network control determines the allocation of scarce resources to manage congestion, a fundamental problem in manufacturing, communications, and healthcare. Compared to standard RL problems, queueing problems are distinguished by unique challenges: i) a system operating in continuous time, ii) high stochasticity, and iii) long horizons over which the system can become unstable (exploding delays). To spur methodological progress tackling these challenges, we present an open-sourced queueing simulation framework, QGym, that benchmark queueing policies across realistic problem instances. Our modular framework allows the researchers to build on our initial instances, which provide a wide range of environments including parallel servers, criss-cross, tandem, and re-entrant networks, as well as a realistically calibrated hospital queuing system. QGym makes it easy to compare multiple policies, including both model-free RL methods and classical queuing policies. Our testbed complements the traditional focus on evaluating algorithms based on mathematical guarantees in idealized settings, and significantly expands the scope of empirical benchmarking in prior work. QGym code is open-sourced at https://github.com/namkoong-lab/QGym.

QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers

TL;DR

An open-sourced queueing simulation framework, QGym, that benchmark queueing policies across realistic problem instances, and makes it easy to compare multiple policies, including both model-free RL methods and classical queuing policies.

Abstract

Queuing network control determines the allocation of scarce resources to manage congestion, a fundamental problem in manufacturing, communications, and healthcare. Compared to standard RL problems, queueing problems are distinguished by unique challenges: i) a system operating in continuous time, ii) high stochasticity, and iii) long horizons over which the system can become unstable (exploding delays). To spur methodological progress tackling these challenges, we present an open-sourced queueing simulation framework, QGym, that benchmark queueing policies across realistic problem instances. Our modular framework allows the researchers to build on our initial instances, which provide a wide range of environments including parallel servers, criss-cross, tandem, and re-entrant networks, as well as a realistically calibrated hospital queuing system. QGym makes it easy to compare multiple policies, including both model-free RL methods and classical queuing policies. Our testbed complements the traditional focus on evaluating algorithms based on mathematical guarantees in idealized settings, and significantly expands the scope of empirical benchmarking in prior work. QGym code is open-sourced at https://github.com/namkoong-lab/QGym.
Paper Structure (24 sections, 8 equations, 5 figures, 3 tables)

This paper contains 24 sections, 8 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work.
  3. An Event-Driven Queuing Simulation Framework
  4. Event-Driven MDP Formulation
  5. Functionality
  6. OpenAI Gym Design.
  7. Benchmark Policies
  8. Traditional Policies
  9. Deep RL based methods
  10. Experiments
  11. Setup
  12. Results
  13. Conclusion
  14. Code release
  15. Additional Benchmark Result
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Highlights of QGym framework for developing and benchmarking queuing algorithms. QGym provides an event-driven simulator and benchmarks a wide range of queuing policies and systems. QGym interface also allows users to easily specify new queuing policies and systems.
  • Figure 2: QGym provides a unified and comprehensive benchmarking system for queueing policies, across a range of realistic environments.
  • Figure 3: Comparison of our work with related methods
  • Figure 4: QGym provides an user-friendly interface to define and run experiments for evaluating routing policies on queuing networks.
  • Figure 5: Queuing systems QGym benchmarks. (a) Real-world example of hospital routing. (b) Real-world example of data input-switch routing. (c) and (f) Two variations of reentrant networks. (d) Five-by-Five netowrk for modeling call centers. (e) Criss-cross network. See details in section \ref{['sec:Experiments']}