Contextual Decision-Making with Knapsacks Beyond the Worst Case

TL;DR

This work studies Contextual Decision-Making with Knapsacks (CDMK), a sequential, budget-constrained problem where the agent observes requests but not external factors each round and distributions are unknown. It introduces a re-solving algorithm that uses empirical estimates of the request and external-factor distributions to compute a fluid-like policy in each round, stopping as resources run out. Under a unique, non-degenerate fluid LP, the method attains an $\tilde{O}(1)$ regret against the fluid benchmark with full information and $\tilde{O}(\log T)$ with partial information, and remains near-optimal in worst cases with $\tilde{O}(\sqrt{T})$ or $\tilde{O}(\sqrt{T}\log T)$ regret, respectively; the results extend to continuous randomness. The paper also shows that a degenerate, unique fluid optimum can cause an $\Omega(\sqrt{T})$ gap between the fluid and online optima, motivating the regularity conditions. Overall, the proposed re-solving framework provides robust performance guarantees across two feedback models and supports practical resource-constrained decision-making in uncertain environments.

Abstract

We study the framework of a dynamic decision-making scenario with resource constraints. In this framework, an agent, whose target is to maximize the total reward under the initial inventory, selects an action in each round upon observing a random request, leading to a reward and resource consumptions that are further associated with an unknown random external factor. While previous research has already established an $\widetilde{O}(\sqrt{T})$ worst-case regret for this problem, this work offers two results that go beyond the worst-case perspective: one for the worst-case gap between benchmarks and another for logarithmic regret rates. We first show that an $Ω(\sqrt{T})$ distance between the commonly used fluid benchmark and the online optimum is unavoidable when the former has a degenerate optimal solution. On the algorithmic side, we merge the re-solving heuristic with distribution estimation skills and propose an algorithm that achieves an $\widetilde{O}(1)$ regret as long as the fluid LP has a unique and non-degenerate solution. Furthermore, we prove that our algorithm maintains a near-optimal $\widetilde{O}(\sqrt{T})$ regret even in the worst cases and extend these results to the setting where the request and external factor are continuous. Regarding information structure, our regret results are obtained under two feedback models, respectively, where the algorithm accesses the external factor at the end of each round and at the end of a round only when a non-null action is executed.

Figures (2)

• Figure 1: An illustration of \ref{['lem:accessing-frequency-partial-info']}. Before $\Theta(T)$ rounds, the probability of accessing frequency at any time step is at least $\Omega(1 / \log T)$, and we have an overall $\Omega(1)$ frequency with high probability.
• Figure 2: Regret of \ref{['alg:re-solving-EE']} under different number of total rounds $T = 2000 \cdot 2^k$ for integer $0 \leq k \leq 5$.

Theorems & Definitions (28)

• Example 1.1: Supply chain management
• Example 1.2: Dynamic bidding in repeated auctions with budgets balseiro2021surveybalseiro2019learning
• Proposition 2.1: balseiro2021survey
• Theorem 2.1: Worst-case gap
• Definition 3.1
• Definition 3.2
• Proposition 3.1: Stability
• Theorem 3.1
• Theorem 4.1
• Lemma 4.1