Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Algorithm Design of Dynamic Spectrum Access by Whittle Index

Keqin Liu, Yiying Zhang, Zhi Ding

TL;DR

This work frames dynamic spectrum access as a restless multi-armed bandit (RMAB) with partial observability via channel quality indicators (CQI) and seeks a low-complexity policy for maximizing the discounted long-run throughput while limiting interference to the parent network. It develops an Approximated Whittle Index (AWI) by proving indexability under a discount factor bound, establishing a threshold structure for the single-armed problem, and deriving a closed-form, iterative index computation $\widehat{W}_n(\omega)$ that can be implemented online. The main contributions are (i) theoretical results on the threshold optimality and indexability under $\beta$ constraints, (ii) a closed-form AWI with a practical $n$-step refinement, and (iii) extensive simulations showing AWI outperforms myopic and rough AWI0 baselines across multiple channel models and settings. The findings offer a scalable, robust approach to dynamic spectrum access in HetNets and 5G-era networks, with potential extensions to richer channel models and data-driven index approximations.

Abstract

This study addresses the dynamic spectrum access problem in a wireless sub-network that shares channels with a parent network. We approach the sequential channel allocation problem using a restless multi-armed bandits (RMAB) framework. Our objective is to maximize the expected discounted return over an infinite horizon while minimizing interference to the parent network caused by shared channels with the sub-network. Due to the unavailability of direct observations of the true channel state, we leverage the channel quality indicator (CQI) feedback provided by users. However, the RMAB problem is widely acknowledged as PSPACE-hard even for finite-state models. To overcome this challenge, we propose a closed-form channel index function using an iterative online approximation method to approximate the well-known Whittle index policy, which offers a low-complexity solution for ranking the available channels that has an infinite state space. Through extensive numerical simulation experiments, we demonstrate the superior performance and robustness of our proposed algorithm.

Efficient Algorithm Design of Dynamic Spectrum Access by Whittle Index

TL;DR

This work frames dynamic spectrum access as a restless multi-armed bandit (RMAB) with partial observability via channel quality indicators (CQI) and seeks a low-complexity policy for maximizing the discounted long-run throughput while limiting interference to the parent network. It develops an Approximated Whittle Index (AWI) by proving indexability under a discount factor bound, establishing a threshold structure for the single-armed problem, and deriving a closed-form, iterative index computation that can be implemented online. The main contributions are (i) theoretical results on the threshold optimality and indexability under constraints, (ii) a closed-form AWI with a practical -step refinement, and (iii) extensive simulations showing AWI outperforms myopic and rough AWI0 baselines across multiple channel models and settings. The findings offer a scalable, robust approach to dynamic spectrum access in HetNets and 5G-era networks, with potential extensions to richer channel models and data-driven index approximations.

Abstract

This study addresses the dynamic spectrum access problem in a wireless sub-network that shares channels with a parent network. We approach the sequential channel allocation problem using a restless multi-armed bandits (RMAB) framework. Our objective is to maximize the expected discounted return over an infinite horizon while minimizing interference to the parent network caused by shared channels with the sub-network. Due to the unavailability of direct observations of the true channel state, we leverage the channel quality indicator (CQI) feedback provided by users. However, the RMAB problem is widely acknowledged as PSPACE-hard even for finite-state models. To overcome this challenge, we propose a closed-form channel index function using an iterative online approximation method to approximate the well-known Whittle index policy, which offers a low-complexity solution for ranking the available channels that has an infinite state space. Through extensive numerical simulation experiments, we demonstrate the superior performance and robustness of our proposed algorithm.
Paper Structure (20 sections, 11 theorems, 91 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 20 sections, 11 theorems, 91 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Dynamic Spectrum Access
  3. Restless Multi-armed Bandit Problem
  4. Related Work
  5. Organization
  6. System Model and Problem Formulation
  7. System Model
  8. Restless Multi-armed Bandit Formulation
  9. Indexability and Threshold Policy
  10. Properties of Value Functions
  11. Optimality of Threshold Policy
  12. Indexability
  13. Approximated Whittle Index
  14. Closed Form of Approximated Whittle Index
  15. Algorithm
  16. ...and 5 more sections

Key Result

Lemma 1

$V_{1,T,\beta,m}(\omega)$ is piecewise linear and convex in both $\omega$ and $m$ for any $T \geq 1$.

Figures (4)

  • Figure 1: The system model
  • Figure 2: Average discounted return using the myopic, AWI0 policies and our AWI policies in different iteration step sizes under the condition (\ref{['eq: 15']}) of discount factor $\beta$
  • Figure 3: Average discounted return using the myopic, AWI0 policies and our AWI policies in different iteration step sizes when discount factor $\beta = 0.9$
  • Figure : Approximated Whittle Index Policy

Theorems & Definitions (11)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Theorem 1
  • Theorem 2
  • Lemma 5
  • Lemma 6
  • Theorem 3
  • Corollary 1
  • ...and 1 more