Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Minimax-MDP Framework with Future-imposed Conditions for Learning-augmented Problems

Xin Chen, Yuze Chen, Yuan Zhou

TL;DR

The paper presents a minimax-MDP framework for learning-augmented sequential decision problems where prediction intervals refine over time and can be adversarially evolved. A central contribution is a set of future-imposed conditions that are both necessary and sufficient for the existence of a feasible policy, enabling robust online decisions with closed-form policies in many cases. The framework is demonstrated across three applications: robust multi-period ordering with predictions, robust resource allocation with predictions, and a multi-phase extension with changing costs, including a phase-reduction approach that reduces K-phase problems to simpler ones. These results yield tractable, prediction-aware robustness guarantees and offer practical policies with linear or polynomial-time computation, significantly advancing robust online decision-making under predictive uncertainty.

Abstract

We study a class of sequential decision-making problems with augmented predictions, potentially provided by a machine learning algorithm. In this setting, the decision-maker receives prediction intervals for unknown parameters that become progressively refined over time, and seeks decisions that are competitive with the hindsight optimal under all possible realizations of both parameters and predictions. We propose a minimax Markov Decision Process (minimax-MDP) framework, where the system state consists of an adversarially evolving environment state and an internal state controlled by the decision-maker. We introduce a set of future-imposed conditions that characterize the feasibility of minimax-MDPs and enable the design of efficient, often closed-form, robustly competitive policies. We illustrate the framework through three applications: multi-period inventory ordering with refining demand predictions, resource allocation with uncertain utility functions, and a multi-phase extension of the minimax-MDP applied to the inventory problem with time-varying ordering costs. Our results provide a tractable and versatile approach to robust online decision-making under predictive uncertainty.

A Minimax-MDP Framework with Future-imposed Conditions for Learning-augmented Problems

TL;DR

The paper presents a minimax-MDP framework for learning-augmented sequential decision problems where prediction intervals refine over time and can be adversarially evolved. A central contribution is a set of future-imposed conditions that are both necessary and sufficient for the existence of a feasible policy, enabling robust online decisions with closed-form policies in many cases. The framework is demonstrated across three applications: robust multi-period ordering with predictions, robust resource allocation with predictions, and a multi-phase extension with changing costs, including a phase-reduction approach that reduces K-phase problems to simpler ones. These results yield tractable, prediction-aware robustness guarantees and offer practical policies with linear or polynomial-time computation, significantly advancing robust online decision-making under predictive uncertainty.

Abstract

We study a class of sequential decision-making problems with augmented predictions, potentially provided by a machine learning algorithm. In this setting, the decision-maker receives prediction intervals for unknown parameters that become progressively refined over time, and seeks decisions that are competitive with the hindsight optimal under all possible realizations of both parameters and predictions. We propose a minimax Markov Decision Process (minimax-MDP) framework, where the system state consists of an adversarially evolving environment state and an internal state controlled by the decision-maker. We introduce a set of future-imposed conditions that characterize the feasibility of minimax-MDPs and enable the design of efficient, often closed-form, robustly competitive policies. We illustrate the framework through three applications: multi-period inventory ordering with refining demand predictions, resource allocation with uncertain utility functions, and a multi-phase extension of the minimax-MDP applied to the inventory problem with time-varying ordering costs. Our results provide a tractable and versatile approach to robust online decision-making under predictive uncertainty.
Paper Structure (47 sections, 23 theorems, 153 equations, 1 figure, 1 table)

This paper contains 47 sections, 23 theorems, 153 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Related Works
  4. The Minimax-MDP Framework
  5. Solving the Minimax-MDPs
  6. Additional Notations
  7. Future-imposed Inventory Level Bounds
  8. Future-imposed Conditions and the Main Equivalence Theorem
  9. Application I: Robust Multi-period Ordering with Predictions
  10. Problem Formulation
  11. Optimizing Robust Regret
  12. Optimizing Robust Competitive Ratio
  13. Application II: Robust Resource Allocation with Predictions
  14. Problem Formulation
  15. The Minimax-MDP-based Solution
  16. ...and 32 more sections

Key Result

Lemma 1

Given a minimax-MDP instance $\mathcal{I}$, if there exists a feasible policy $\pi$, then for any $t \in \{1,2,\dots,T\}$ and $\pi$-compatible (partial) trajectory $(s_1, x_1, s_2, x_2, \dots, s_t, x_t)$, we have that $L_{\star \leadsto t}(s_t) \leq x_t \leq R_{\star \leadsto t}(s_t)$.

Figures (1)

  • Figure EC.1: Illustration of an ordering policy under the single-switching prediction sequences.

Theorems & Definitions (33)

  • Definition 1
  • Definition 2
  • Lemma 1: Future-imposed inventory bounds
  • Lemma 2: Strengthened future-imposed inventory bounds
  • Definition 3: Future-imposed conditions
  • Lemma 3
  • Theorem 1: Equivalent conditions.
  • Claim 1
  • Claim 2
  • Proposition 1
  • ...and 23 more