OMGPT: A Sequence Modeling Framework for Data-driven Operational Decision Making

TL;DR

OMGPT reframes sequential decision making as sequence modeling and trains a Generative Pre-trained Transformer to predict optimal actions from history across diverse, data-generated environments. By pre-training on broad environment classes and leveraging Bayes-inspired analysis, OMGPT achieves sub-linear regret and strong generalization without relying on a fixed analytical model. The work demonstrates superior empirical performance across dynamic pricing, inventory, queuing, and revenue management tasks, and provides theoretical foundations linking pre-training diversity, environment inference, and decision quality. Its results suggest a practical, scalable, and robust paradigm for data-driven operational decision making with broad applicability and interpretability.

Abstract

We build a Generative Pre-trained Transformer (GPT) model from scratch to solve sequential decision making tasks arising in contexts of operations research and management science which we call OMGPT. We first propose a general sequence modeling framework to cover several operational decision making tasks as special cases, such as dynamic pricing, inventory management, resource allocation, and queueing control. Under the framework, all these tasks can be viewed as a sequential prediction problem where the goal is to predict the optimal future action given all the historical information. Then we train a transformer-based neural network model (OMGPT) as a natural and powerful architecture for sequential modeling. This marks a paradigm shift compared to the existing methods for these OR/OM tasks in that (i) the OMGPT model can take advantage of the huge amount of pre-trained data; (ii) when tackling these problems, OMGPT does not assume any analytical model structure and enables a direct and rich mapping from the history to the future actions. Either of these two aspects, to the best of our knowledge, is not achieved by any existing method. We establish a Bayesian perspective to theoretically understand the working mechanism of the OMGPT on these tasks, which relates its performance with the pre-training task diversity and the divergence between the testing task and pre-training tasks. Numerically, we observe a surprising performance of the proposed model across all the above tasks.

Figures (25)

• Figure 1: The pre-training phase of OMGPT. We first generate different environments $\gamma_i$. For each $\gamma_i$, we generate the corresponding histories and optimal actions. The histories serve as the inputs to the OMGPT, which then uses them to predict the corresponding optimal actions. The goal of the pre-training is to minimize the empirical prediction errors.
• Figure 2: The testing/application phase of the pre-trained OMGPT. we iteratively input the history $H_t$ into the pre-trained OMGPT $\texttt{TF}_{\hat{\theta}}$ to obtain an action $a_t$. This action is then applied to interact with the testing environment $\gamma$, generating a new observation $O_t$. The cycle is then repeated with the updated history $H_{t+1}$.
• Figure 3: $\texttt{TF}_{\hat{\theta}}$ nearly matches the optimal decision function $\texttt{Alg}^*$ in dynamic pricing tasks with concentrated deviations. Figure (a) shows one decision trial for $\texttt{Alg}^*$ and $\texttt{TF}_{\hat{\theta}}$ ,where the optimal actions change over time because $X_t$'s are different for different time $t$. Figure (b) shows the histogram of $\texttt{TF}_{\hat{\theta}}-\texttt{Alg}^*$ over different environments. The experiment setup and more results are deferred to Appendix \ref{['appx:bayes_match_exp']}.
• Figure 4: The average out-of-sample regret (first row) and action suboptimality, i.e., $|a^*_t - \texttt{Alg}(H_t)|$, (second row) of $\texttt{TF}_{\hat{\theta}}$ against benchmark algorithms (see details in Appendix \ref{['appx:benchmark']}). The numbers in the legend bar are the final cumulative regret. The last two tasks can have negative regret during the horizon since they are MDP problems where the optimal actions may achieve a lower single-period reward than the applied algorithms, although they maximize the final cumulative expected reward. For revenue management, since the exact optimal actions are computed at a high computation cost, we use the actions from the Adaptive Allocation Algorithm (Ada) from chen2024improved, which can achieve constant regret, to approximate optimal actions in both the pre-training and testing (and thus not show the suboptimality of the Ada algorithm in (h)), and use their upper bound of the optimal cumulative reward to compute (an upper bound) of the regret used in (d).
• Figure 5: The advantage of OMGPT compared to the best benchmark algorithm in dynamic pricing, across different model size (number of layers) and problem complexity (problem dimensions).

Theorems & Definitions (26)

• Proposition 5.1
• Corollary 5.2
• Proposition 5.3: Surrogate property
• Proposition 5.4
• Theorem 5.5: Regret Upper Bound (Informal)
• Theorem 5.9: Regret Upper Bound
• Example 5.10
• Theorem 5.11: Regret Upper Bound on Finite Environment Space
• Example 5.12
• Claim B.1