LookAhead: The Optimal Non-decreasing Index Policy for a Time-Varying Holding Cost problem

TL;DR

This paper tackles the Time-Varying Holding Cost problem in a two-class M/M/1 queue, where class 1 costs grow with age and class 2 costs remain constant. It derives the first optimal (non-decreasing) index policy, named LookAhead, by choosing the cost to optimize some lookahead time $X$ into the future and showing the optimal $X$ (with class 1 index $V_1(t)=\mu_1\mathbb{E}[c_1(t+X)], X\sim\text{Exp}(\mu_1-\lambda_1)$, and class 2 index $V_2(t)=\mu_2 c_2$). The authors connect LookAhead to Whittle index formulations for the corresponding restless bandit problem and provide both a shorter proof under an additional assumption and a longer, assumption-free proof, along with a detailed simulation study demonstrating significant performance gains over standard heuristics. The work delivers finite-time optimality results in a challenging TVHC setting and offers amortized cost interpretations and guidance for broader TVHC problems. This advances both theoretical understanding and practical scheduling in environments with aging delay costs.

Abstract

In practice, the cost of delaying a job can grow as the job waits. Such behavior is modeled by the Time-Varying Holding Cost (TVHC) problem, where each job's instantaneous holding cost increases with its current age (a job's age is the time since it arrived). The goal of the TVHC problem is to find a scheduling policy that minimizes the time-average total holding cost across all jobs. However, no optimality results are known for the TVHC problem outside of the asymptotic regime. In this paper, we study a simple yet still challenging special case: A two-class M/M/1 queue in which class 1 jobs incur a non-decreasing, time-varying holding cost and class 2 jobs incur a constant holding cost. Our main contribution is deriving the first optimal (non-decreasing) index policy for this special case of the TVHC problem. Our optimal policy, called LookAhead, stems from the following idea: Rather than considering each job's current holding cost when making scheduling decisions, we should look at their cost some $X$ time into the future, where this $X$ is intuitively called the ``lookahead amount." This paper derives that optimal lookahead amount.

Figures (8)

• Figure 1: A 2-class M/M/1 Queue with Age-based Holding Costs.
• Figure 2: (Instantaneous) Holding cost and Cumulative Holding Cost.
• Figure 3: $\operatorname{Overtake}(\alpha)$ Policy.
• Figure 4: Expected Additional Cost accrued by a class 1 jobs which has reached age $t$.
• Figure 5: Only jobs in the active queue (on the right) are visible to the server.

Theorems & Definitions (42)

• Theorem 1.1: LookAhead
• Lemma 3.2
• proof
• Lemma 3.3: Tail Probability of Class 1 jobs
• proof
• Definition 3.4: $\widehat{Cost}$
• Lemma 3.5
• proof
• Lemma 3.6
• proof