Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Learning in Markovian Restless Bandits over Interference Graphs for Stable Spectrum Sharing

Liad Lea Didi, Kobi Cohen

TL;DR

This work tackles distributed spectrum access in interference graphs with multiple cells and unknown restless Markov channels. It introduces SMILE, a three-phase algorithm (Exploration, Allocation, Exploitation) that learns channel statistics and computes a generalized Gale–Shapley stable allocation in a fully distributed fashion. Theoretical analysis proves logarithmic regret relative to an oracle with full knowledge of expected utilities, and simulations confirm rapid convergence and strong performance against baselines. The model and results advance scalable, interference-aware spectrum sharing in cognitive networks where channel dynamics are heterogeneous and unknown.

Abstract

We study distributed learning for spectrum access and sharing among multiple cognitive communication entities, such as cells, subnetworks, or cognitive radio users (collectively referred to as cells), in communication-constrained wireless networks modeled by interference graphs. Our goal is to achieve a globally stable and interference-aware channel allocation. Stability is defined through a generalized Gale-Shapley multi-to-one matching, a well-established solution concept in wireless resource allocation. We consider wireless networks where L cells share S orthogonal channels and cannot simultaneously use the same channel as their neighbors. Each channel evolves as an unknown restless Markov process with cell-dependent rewards, making this the first work to establish global Gale-Shapley stability for channel allocation in a stochastic, temporally varying restless environment. To address this challenge, we develop SMILE (Stable Multi-matching with Interference-aware LEarning), a communication-efficient distributed learning algorithm that integrates restless bandit learning with graph-constrained coordination. SMILE enables cells to distributedly balance exploration of unknown channels with exploitation of learned information. We prove that SMILE converges to the optimal stable allocation and achieves logarithmic regret relative to a genie with full knowledge of expected utilities. Simulations validate the theoretical guarantees and demonstrate SMILE's robustness, scalability, and efficiency across diverse spectrum-sharing scenarios.

Distributed Learning in Markovian Restless Bandits over Interference Graphs for Stable Spectrum Sharing

TL;DR

This work tackles distributed spectrum access in interference graphs with multiple cells and unknown restless Markov channels. It introduces SMILE, a three-phase algorithm (Exploration, Allocation, Exploitation) that learns channel statistics and computes a generalized Gale–Shapley stable allocation in a fully distributed fashion. Theoretical analysis proves logarithmic regret relative to an oracle with full knowledge of expected utilities, and simulations confirm rapid convergence and strong performance against baselines. The model and results advance scalable, interference-aware spectrum sharing in cognitive networks where channel dynamics are heterogeneous and unknown.

Abstract

We study distributed learning for spectrum access and sharing among multiple cognitive communication entities, such as cells, subnetworks, or cognitive radio users (collectively referred to as cells), in communication-constrained wireless networks modeled by interference graphs. Our goal is to achieve a globally stable and interference-aware channel allocation. Stability is defined through a generalized Gale-Shapley multi-to-one matching, a well-established solution concept in wireless resource allocation. We consider wireless networks where L cells share S orthogonal channels and cannot simultaneously use the same channel as their neighbors. Each channel evolves as an unknown restless Markov process with cell-dependent rewards, making this the first work to establish global Gale-Shapley stability for channel allocation in a stochastic, temporally varying restless environment. To address this challenge, we develop SMILE (Stable Multi-matching with Interference-aware LEarning), a communication-efficient distributed learning algorithm that integrates restless bandit learning with graph-constrained coordination. SMILE enables cells to distributedly balance exploration of unknown channels with exploitation of learned information. We prove that SMILE converges to the optimal stable allocation and achieves logarithmic regret relative to a genie with full knowledge of expected utilities. Simulations validate the theoretical guarantees and demonstrate SMILE's robustness, scalability, and efficiency across diverse spectrum-sharing scenarios.

Paper Structure

This paper contains 16 sections, 1 theorem, 26 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Main Results
  3. Other Related Work
  4. Network Model and Problem Statement
  5. Network Model
  6. Notation
  7. Stable Multi-Matching Formulation
  8. Objective
  9. The SMILE Algorithm
  10. The Exploration Phase
  11. The Exploration Function
  12. The Access Phase (Allocation)
  13. Distributed Implementation and Illustrative Example
  14. The Stable Spectrum Sharing Phase (Exploitation)
  15. Theoretical Regret Analysis
  16. ...and 1 more sections

Key Result

Theorem 1

Assuming that the proposed SMILE algorithm is implemented and that the assumptions on the system model in Section sec:problem hold. Then, the regret at time $t$ is upper bounded by:

Figures (5)

  • Figure 1: An illustration of the SMILE algorithm phases for cell $\ell$.
  • Figure 2: Interference graph topology of 5 cells and 3 channels. Edges indicate interference between cells.
  • Figure 3: Estimated expected rate matrix [Mbps].
  • Figure 4: Allocation phase iterations.
  • Figure :

Theorems & Definitions (2)

  • Definition 1: Generalized Gale-Shapley Stable Allocations ami2024stable
  • Theorem 1