Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contextual Bandits with Non-Stationary Correlated Rewards for User Association in MmWave Vehicular Networks

Xiaoyang He, Xiaoxia Huang, Lanhua Li

TL;DR

Numerical results show that the proposed SD-CC-UCB algorithm achieves the network throughput within 100%–103% of a benchmark algorithm which requires perfect instantaneous CSI, demonstrating the effectiveness of SD-CC-UCB in vehicular communications.

Abstract

Millimeter wave (mmWave) communication has emerged as a propelling technology in vehicular communication. Usually, an appropriate decision on user association requires timely channel information between vehicles and base stations (BSs), which is challenging given a fast-fading mmWave vehicular channel. In this paper, relying solely on learning transmission rate, we propose a low-complexity semi-distributed contextual correlated upper confidence bound (SD-CC-UCB) algorithm to establish an up-to-date user association without explicit measurement of channel state information (CSI). Under a contextual multi-arm bandits framework, SD-CC-UCB learns and predicts the transmission rate given the location and velocity of the vehicle, which can adequately capture the intricate channel condition for a prompt decision on user association. Further, SD-CC-UCB efficiently identifies the set of candidate BSs which probably support supreme transmission rate by leveraging the correlated distributions of transmission rates on different locations. To further refine the learning transmission rate over the link to candidate BSs, each vehicle deploys the Thompson Sampling algorithm by taking the interference among vehicles and handover overhead into consideration. Numerical results show that our proposed algorithm achieves the network throughput within 100%-103% of a benchmark algorithm which requires perfect instantaneous CSI, demonstrating the effectiveness of SD-CC-UCB in vehicular communications.

Contextual Bandits with Non-Stationary Correlated Rewards for User Association in MmWave Vehicular Networks

TL;DR

Numerical results show that the proposed SD-CC-UCB algorithm achieves the network throughput within 100%–103% of a benchmark algorithm which requires perfect instantaneous CSI, demonstrating the effectiveness of SD-CC-UCB in vehicular communications.

Abstract

Millimeter wave (mmWave) communication has emerged as a propelling technology in vehicular communication. Usually, an appropriate decision on user association requires timely channel information between vehicles and base stations (BSs), which is challenging given a fast-fading mmWave vehicular channel. In this paper, relying solely on learning transmission rate, we propose a low-complexity semi-distributed contextual correlated upper confidence bound (SD-CC-UCB) algorithm to establish an up-to-date user association without explicit measurement of channel state information (CSI). Under a contextual multi-arm bandits framework, SD-CC-UCB learns and predicts the transmission rate given the location and velocity of the vehicle, which can adequately capture the intricate channel condition for a prompt decision on user association. Further, SD-CC-UCB efficiently identifies the set of candidate BSs which probably support supreme transmission rate by leveraging the correlated distributions of transmission rates on different locations. To further refine the learning transmission rate over the link to candidate BSs, each vehicle deploys the Thompson Sampling algorithm by taking the interference among vehicles and handover overhead into consideration. Numerical results show that our proposed algorithm achieves the network throughput within 100%-103% of a benchmark algorithm which requires perfect instantaneous CSI, demonstrating the effectiveness of SD-CC-UCB in vehicular communications.
Paper Structure (16 sections, 34 equations, 9 figures, 1 table, 4 algorithms)

This paper contains 16 sections, 34 equations, 9 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. System Model
  4. Mobility Model
  5. Channel Model
  6. Optimization Problem and Algorithm Architecture
  7. Contextual Correlated Upper Confidence Bound Algorithm
  8. Contextual Multi-Armed Bandits
  9. Contextual Correlated UCB Algorithm
  10. Regret Analysis
  11. Complexity Analysis
  12. Semi-distributed Contextual Correlated Upper Confidence Bound Algorithm
  13. SD-CC-UCB Algorithm
  14. Complexity and Regret Analysis
  15. Numerical Results
  16. ...and 1 more sections

Figures (9)

  • Figure 1: A multi-user multi-BSs vehicular network, where vehicles within the same context experience a comparable channel condition.
  • Figure 2: Architecture of SD-CC-UCB algorithm.
  • Figure 3: The grids represent the location dimension of contexts. $p$ and $R_{max}$ are the infraction probability and the highest achievable transmission rate for a vehicle, respectively. $\mathbb{\hat{\phi}}_{D,j}$ indicates the lowest empirical contextual pseudo-reward of arm $j$ on context $D$.
  • Figure 4: Cumulative regret.
  • Figure 5: Error probability of CC-UCB identifying non-competitive arms and the ratio of non-competitive arms to all arms.
  • ...and 4 more figures