Bi-Level Online Provisioning and Scheduling with Switching Costs and Cross-Level Constraints

TL;DR

This work addresses online bandwidth provisioning and stateful queue scheduling across two time scales, formulating it as a bi-level problem where upper-level OCO with switching costs selects budgets and lower-level CMDP controls fast-time-scale decisions. The authors propose Bi-Level Optimization and Learning (BLOL) with a dual-feedback mechanism: the lower level provides a budget-sensitive dual multiplier that guides the upper-level subgradient updates, while an extended occupancy-measure LP yields both optimal lower-level policies and the dual signal. They prove near-optimal regret bounds and high-probability safety guarantees, balancing exploration and feasibility under dynamic budgets via pessimistic safety margins and budget-aware optimism. Numerical results on synthetic and real traffic traces demonstrate that the method outperforms fixed-budget and decoupled baselines, achieving lower objective gaps while maintaining tight budget compliance. The framework promises practical impact for network slicing and edge computing, enabling efficient, safe resource provisioning under uncertain dynamics and reconfiguration costs.

Abstract

We study a bi-level online provisioning and scheduling problem motivated by network resource allocation, where provisioning decisions are made at a slow time scale while queue-/state-dependent scheduling is performed at a fast time scale. We model this two-time-scale interaction using an upper-level online convex optimization (OCO) problem and a lower-level constrained Markov decision process (CMDP). Existing OCO typically assumes stateless decisions and thus cannot capture MDP network dynamics such as queue evolution. Meanwhile, CMDP algorithms typically assume a fixed constraint threshold, whereas in provisioning-and-scheduling systems, the threshold varies with online budget decisions. To address these gaps, we study bi-level OCO-CMDP learning under switching costs (budget reprovisioning/system reconfiguration) and cross-level constraints that couple budgets to scheduling decisions. Our new algorithm solves this learning problem via several non-trivial developments, including a carefully designed dual feedback that returns the budget multiplier as sensitivity information for the upper-level update and a lower level that solves a budget-adaptive safe exploration problem via an extended occupancy-measure linear program. We establish near-optimal regret and high-probability satisfaction of the cross-level constraints.

Figures (3)

• Figure 1: Online bandwidth provisioning with queue-aware scheduling.
• Figure 2: Coupled two-time-scale decision making: at the beginning of episode $k$, the agent provisions a resource budget $b_k$; within the episode, it follows a state-dependent policy selecting actions $a_{k,t}$ over $T$ time slots, incurring losses that aggregate to the episode cost and resource usage that must satisfy the cross-level budget constraint.
• Figure 3: Cumulative objective gap and cumulative budget violation under (a,b) Poisson arrivals and (c,d) MAWI traces.

Theorems & Definitions (7)

• Lemma 1: Lower-level BALDE regret
• proof : Proof sketch of Lemma \ref{['thm:bdope_main']}
• Lemma 2: Upper-level BLOL Regret with Switching Costs
• proof : Proof Sketch of Lemma \ref{['lem:oco_regret']}
• Theorem 1: Total regret of our bi-level algorithm BLOL
• Theorem 2: Safety Guarantee
• proof : Proof sketch of Theorem \ref{['safety_guarantee']}