Learning an Inventory Control Policy with General Inventory Arrival Dynamics

TL;DR

The paper addresses inventory control under general, multi-shipment arrival dynamics (QOT) and downstream post-processing of orders, advancing beyond fixed lead-time assumptions. It formulates the problem as an exogenous decision process (IDP), and introduces Gen-QOT to learn and simulate realistic arrivals while a differentiable simulator enables policy learning via supervised-like backtests. Key contributions include the Gen-QOT arrivals model, the integration of a post-processor for vendor constraints, and extensive empirical validation through backtests and real-world A/B tests, demonstrating improved profitability and robust off-policy generalization. The work has practical significance for real supply chains by enabling data-driven, scalable RL policies that adapt to complex supplier behaviors and constraint structures.

Abstract

In this paper we address the problem of learning and backtesting inventory control policies in the presence of general arrival dynamics -- which we term as a quantity-over-time arrivals model (QOT). We also allow for order quantities to be modified as a post-processing step to meet vendor constraints such as order minimum and batch size constraints -- a common practice in real supply chains. To the best of our knowledge this is the first work to handle either arbitrary arrival dynamics or an arbitrary downstream post-processing of order quantities. Building upon recent work (Madeka et al., 2022) we similarly formulate the periodic review inventory control problem as an exogenous decision process, where most of the state is outside the control of the agent. Madeka et al., 2022 show how to construct a simulator that replays historic data to solve this class of problem. In our case, we incorporate a deep generative model for the arrivals process as part of the history replay. By formulating the problem as an exogenous decision process, we can apply results from Madeka et al., 2022 to obtain a reduction to supervised learning. Via simulation studies we show that this approach yields statistically significant improvements in profitability over production baselines. Using data from a real-world A/B test, we show that Gen-QOT generalizes well to off-policy data and that the resulting buying policy outperforms traditional inventory management systems in real world settings.

Figures (12)

• Figure 1: Orders often arrive in multiple shipments, and spread out over multiple time periods.
• Figure 2: Example of yields -- multiplicative proportion of order quantity that is received -- for purchase orders at a large e-retailer, both before and after any post-processor is applied to the order quantity.
• Figure 3: Cumulative arrivals over time for a single purchase order under differing dynamics.
• Figure 4: A set of 256 real and simulated sample paths of order-quantity normalized inventory arrivals.
• Figure 5: Residual calibration plot for cumulative inventory received -- residuals are plotted against predicted values in the main figure, while the original plot is shown in an inner figure.

Theorems & Definitions (1)

• Remark 3.2: Forecasting Arrivals