Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Robustness of Age for Learning-Based Wireless Scheduling in Unknown Environments

Juaren Steiger, Bin Li

TL;DR

This paper tackles constrained CMAB-based wireless scheduling under unknown channel states, introducing an age-based approach that uses the head-of-line age $Z_{k,t}^\to$ instead of the virtual queue length to enforce short-term throughput constraints over a window $W$. The authors design an Age-Based Bandit Learning Policy that combines UCB with an age-weighted drift term, achieving zero throughput violation for $W$ on the order of $O(\eta/\varepsilon)$ and a regret bound $O(\varepsilon T + T/\eta + \sqrt{T\log T})$ in i.i.d. environments. Theoretical analysis relies on Lyapunov drift techniques with two Lyapunov functions and a drift lemma to handle the age dynamics, while empirical results show comparable performance to state-of-the-art methods under stationarity and superior robustness under abrupt changes. The findings suggest that using head-of-line age yields more stable and rapid recovery from constraint infeasibility, enabling more reliable online wireless scheduling in unknown and nonstationary environments.

Abstract

The constrained combinatorial multi-armed bandit model has been widely employed to solve problems in wireless networking and related areas, including the problem of wireless scheduling for throughput optimization under unknown channel conditions. Most work in this area uses an algorithm design strategy that combines a bandit learning algorithm with the virtual queue technique to track the throughput constraint violation. These algorithms seek to minimize the virtual queue length in their algorithm design. However, in networks where channel conditions change abruptly, the resulting constraints may become infeasible, leading to unbounded growth in virtual queue lengths. In this paper, we make the key observation that the dynamics of the head-of-line age, i.e. the age of the oldest packet in the virtual queue, make it more robust when used in algorithm design compared to the virtual queue length. We therefore design a learning-based scheduling policy that uses the head-of-line age in place of the virtual queue length. We show that our policy matches state-of-the-art performance under i.i.d. network conditions. Crucially, we also show that the system remains stable even under abrupt changes in channel conditions and can rapidly recover from periods of constraint infeasibility.

On the Robustness of Age for Learning-Based Wireless Scheduling in Unknown Environments

TL;DR

This paper tackles constrained CMAB-based wireless scheduling under unknown channel states, introducing an age-based approach that uses the head-of-line age instead of the virtual queue length to enforce short-term throughput constraints over a window . The authors design an Age-Based Bandit Learning Policy that combines UCB with an age-weighted drift term, achieving zero throughput violation for on the order of and a regret bound in i.i.d. environments. Theoretical analysis relies on Lyapunov drift techniques with two Lyapunov functions and a drift lemma to handle the age dynamics, while empirical results show comparable performance to state-of-the-art methods under stationarity and superior robustness under abrupt changes. The findings suggest that using head-of-line age yields more stable and rapid recovery from constraint infeasibility, enabling more reliable online wireless scheduling in unknown and nonstationary environments.

Abstract

The constrained combinatorial multi-armed bandit model has been widely employed to solve problems in wireless networking and related areas, including the problem of wireless scheduling for throughput optimization under unknown channel conditions. Most work in this area uses an algorithm design strategy that combines a bandit learning algorithm with the virtual queue technique to track the throughput constraint violation. These algorithms seek to minimize the virtual queue length in their algorithm design. However, in networks where channel conditions change abruptly, the resulting constraints may become infeasible, leading to unbounded growth in virtual queue lengths. In this paper, we make the key observation that the dynamics of the head-of-line age, i.e. the age of the oldest packet in the virtual queue, make it more robust when used in algorithm design compared to the virtual queue length. We therefore design a learning-based scheduling policy that uses the head-of-line age in place of the virtual queue length. We show that our policy matches state-of-the-art performance under i.i.d. network conditions. Crucially, we also show that the system remains stable even under abrupt changes in channel conditions and can rapidly recover from periods of constraint infeasibility.
Paper Structure (20 sections, 10 theorems, 78 equations, 6 figures, 1 table, 1 algorithm)

This paper contains 20 sections, 10 theorems, 78 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. Slater's Condition
  4. Age-Based Bandit Learning Algorithm Design
  5. The Queue of delivery requests and the Head-of-Line Age
  6. Throughput for Any Policy in Terms of Head-of-Line Age
  7. Throughput Violation Bounds for Age-Based Policies
  8. An Age-Based Bandit Learning Policy
  9. Performance of the Age-Based Bandit Learning Policy in an I.I.D. Environment
  10. Empirical Performance
  11. Theoretical Analysis
  12. Robustness of the Age-Based Bandit Learning Policy under Abruptly Changing Conditions
  13. Experimental Setup
  14. Discussion of Results
  15. Replication of the Experiment Using a Real Data Trace
  16. ...and 5 more sections

Key Result

Lemma 1

For any causal policy $\pi$ that is a feasible solution to eq:main_problem, there is a corresponding static policy $\sigma$ defined by $\sigma(a) = \frac{1}{T} \sum_{t=1}^T \pi_{t}(a)$ for all actions ${a \in \mathcal{A}}$ that achieves for all timeslots $\tau$, and (ii) $\mathop{\mathbb{E}}\mleft[ R_{k}(S_\tau,A_\tau^\sigma) \mright] \geq \chi_k$ for all $k \in [K]$ and timeslots $\tau$.

Figures (6)

  • Figure 1: An agricultural remote monitoring system.
  • Figure 2: The virtual queue $\mathcal{Q}_{k,t}$ of delivery requests for arm $k$ in timeslot $t$.
  • Figure 3: Result of the simulations from Section \ref{['section:empirical_performance']} (1000 simulation runs).
  • Figure 4: Comparison of policies for short-term throughput and freshness (TSLR) when the throughput constraint for arm $k=1$ becomes abruptly infeasible (shaded region). 1000 simulation runs are performed.
  • Figure 5: Operation of the queue-length-based policy vs. our age-based policy \ref{['eq:age_based_algorithm_def']} when the throughput constraint for arm $k=1$ is infeasible. Here, one simulation run from the results in Fig. \ref{['fig:infeasible_constraints_overall']} is shown.
  • ...and 1 more figures

Theorems & Definitions (25)

  • Definition 1: causal policy
  • Definition 2: static policy
  • Lemma 1
  • proof
  • Lemma 2
  • Lemma 3
  • proof
  • Definition 3: age-based policy
  • Remark 1: arrival independence of age-based policies
  • Theorem 1
  • ...and 15 more