Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Whittle Index Based User Association in Dense Millimeter Wave Networks

Mandar R. Nalavade, Gaurav S. Kasbekar, Vivek S. Borkar

TL;DR

This work tackles scalable user association in dense mmWave networks by formulating the problem as a restless multi-armed bandit and applying Whittle's index relaxation to decouple per-BS queues into independent MDPs. It proves Whittle indexability, provides a method to compute Whittle indices, and proposes an index-based association that assigns each arriving user to the mBS with the smallest index. Empirical results via simulations show that the Whittle index policy outperforms several heuristic baselines in average cost, delay, throughput, and Jain's fairness. The approach offers a principled, scalable strategy for load balancing and delay reduction in ultra-dense mmWave deployments, with potential extensions to multi-channel and frequency-selective environments.

Abstract

We address the problem of user association in a dense millimeter wave (mmWave) network, in which each arriving user brings a file containing a random number of packets and each time slot is divided into multiple mini-slots. This problem is an instance of the restless multi-armed bandit problem, and is provably hard to solve. Using a technique introduced by Whittle, we relax the hard per-stage constraint that each arriving user must be associated with exactly one mmWave base station (mBS) to a long-term constraint and then use the Lagrangian multiplier technique to convert the problem into an unconstrained problem. This decouples the process governing the system into separate Markov Decision Processes at different mBSs. We prove that the problem is Whittle indexable, present a scheme for computing the Whittle indices of different mBSs, and propose an association scheme under which, each arriving user is associated with the mBS with the smallest value of the Whittle index. Using extensive simulations, we show that the proposed Whittle index based scheme outperforms several user association schemes proposed in prior work in terms of various performance metrics such as average cost, delay, throughput, and Jain's fairness index.

Whittle Index Based User Association in Dense Millimeter Wave Networks

TL;DR

This work tackles scalable user association in dense mmWave networks by formulating the problem as a restless multi-armed bandit and applying Whittle's index relaxation to decouple per-BS queues into independent MDPs. It proves Whittle indexability, provides a method to compute Whittle indices, and proposes an index-based association that assigns each arriving user to the mBS with the smallest index. Empirical results via simulations show that the Whittle index policy outperforms several heuristic baselines in average cost, delay, throughput, and Jain's fairness. The approach offers a principled, scalable strategy for load balancing and delay reduction in ultra-dense mmWave deployments, with potential extensions to multi-channel and frequency-selective environments.

Abstract

We address the problem of user association in a dense millimeter wave (mmWave) network, in which each arriving user brings a file containing a random number of packets and each time slot is divided into multiple mini-slots. This problem is an instance of the restless multi-armed bandit problem, and is provably hard to solve. Using a technique introduced by Whittle, we relax the hard per-stage constraint that each arriving user must be associated with exactly one mmWave base station (mBS) to a long-term constraint and then use the Lagrangian multiplier technique to convert the problem into an unconstrained problem. This decouples the process governing the system into separate Markov Decision Processes at different mBSs. We prove that the problem is Whittle indexable, present a scheme for computing the Whittle indices of different mBSs, and propose an association scheme under which, each arriving user is associated with the mBS with the smallest value of the Whittle index. Using extensive simulations, we show that the proposed Whittle index based scheme outperforms several user association schemes proposed in prior work in terms of various performance metrics such as average cost, delay, throughput, and Jain's fairness index.
Paper Structure (17 sections, 9 theorems, 66 equations, 5 figures, 4 tables)

This paper contains 17 sections, 9 theorems, 66 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Model And Problem Formulation
  4. Stability Analysis
  5. Dynamic Programming Equation
  6. Structural Properties of Value Function
  7. Threshold Behavior of Optimal Policy
  8. Whittle Indexability
  9. Whittle Index Computation
  10. Other Existing Policies
  11. Random Policy
  12. Load Based Policy
  13. SNR Based Policy
  14. Throughput Based Policy
  15. Mixed Policy
  16. ...and 2 more sections

Key Result

Theorem 1

If $Lr_K > \sum_{j=0}^M j p_j$, then the induced DTMC of each mBS $i$ is positive recurrent.

Figures (5)

  • Figure 1: The figure shows an example of the system model with $K = 5$ mBSs, serving different numbers of packets. A user arrives with a file size of $j$ packets w.p. $p_j$.
  • Figure 2: The figures compare the average costs achieved under the six association policies. The following parameter values are used for both the plots: $K=5$, $M=100$, $L=20$, $r=[0.78,0.65,0.56,0.50,0.45]$, $p_0 = 0.6$, $p_j = 0.004,\; \forall j \in \{1,2,\ldots,M\}$. Also, for figure (a) (respectively, (b)), the cost vector $C=[95,75,58,40,32]$, (respectively, $C=[32,40,58,75,95]$) is used.
  • Figure 3: The figures compare the average costs achieved under the six association policies. The following parameter values are used for both the plots: $K=8$, $r=[0.78, 0.70, 0.65, 0.60, 0.56, 0.50, 0.48, 0.45]$. The following parameter values are used for figure (a): $M=100$, $L=20$, $p_0 = 0.4$, $p_j = 0.006,\; \forall j \in \{1,2,\ldots,M\}$, $C=[95, 80, 72, 65, 58, 47, 40, 32]$. The following parameter values are used for figure (b): $M=150$, $L=10$, $p_0 = 0.7$$p_j = 0.002,\; \forall j \in \{1,2,\ldots,M\}$, $C=[85, 75, 68, 63, 57, 49, 45, 36]$.
  • Figure 4: The figures compare the average costs achieved under the six association policies. The maximum number of arrivals used for both the plots is: $M=100$. The parameter values used for figure (a) are: $K=10$, $L=15$, $p_{0} = 0.4$, $p_j = 0.006,\; \forall j \in \{1,2,\ldots,M\}$, $r=[0.78,0.75,0.70,0.65,0.58,0.52,0.48,0.46,0.44,0.42]$, $C=[95,85,75,65,58,47,40,32,28,25]$. The parameter values used for figure (b) are: $K=5$, $p_{0} = 0.6$, $p_j = 0.004,\; \forall j \in \{1,2,\ldots,M\}$, $r=[0.78,0.65,0.56,0.50,0.45]$, $C=[95,75,58,40,32]$. Also, in figure (b), $L$ varies from $20$ to $120$.
  • Figure 5: The figures compare the average costs achieved under the six association policies. The values $L=30$ and $p_{0} = 0.8$ are used for both the figures. The following parameter values are used for figure (a): $K=6$, $p_j = 0.2/M,\; \forall j \in \{1,2,\ldots,M\}$, $r=[0.78,0.70,0.65,0.60,0.52,0.46]$, $C=[92,81,70,63,52,40]$, buffer size $= 250$ and different $M$ varying from $100$ to $200$. The following parameter values are used for figure (b): $M=100$, $p_j = 0.002,\; \forall j \in \{1,2,\ldots,M\}$, and different $K$ varying from $5$ to $15$. For $K=5$, the following parameter values are used: $r = [0.78,0.75,0.72,0.69,0.66]$ and $C = [90,86,82,78,74]$. For every subsequent addition of the $i^{th}$ mBS, the values of $r_i$ and $C_i$ are selected as $0.81-0.3i$ and $94-4i$, respectively, where $i \in \{6,7,\ldots,15\}$.

Theorems & Definitions (10)

  • Remark 1
  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Lemma 7
  • Theorem 2