Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Requesting in Decentralized Edge Networks via Non-Stationary Bandits

Yi Zhuang, Kun Yang, Xingran Chen

TL;DR

This work tackles the problem of minimizing information freshness in decentralized edge networks where multiple clients route requests through edge gateways with limited coordination. By reframing the problem as a non-stationary Aging Bandit with history-dependent, correlated rewards, the authors introduce ABAR, a decentralized algorithm that combines adaptive windowing with an adaptive reset mechanism and periodic monitoring to track evolving reward dynamics. They provide a theoretical framework showing sublinear AoI regret and asymptotic optimality under mixed abrupt and gradual changes, extending drift-change analysis to decentralized, correlated settings. Simulations demonstrate that ABAR approaches the performance of a centralized oracle and outperforms classical decentralized bandit baselines across challenging scenarios, highlighting its practical impact for scalable, low-latency information delivery in edge networks.

Abstract

We study a decentralized collaborative requesting problem that aims to optimize the information freshness of time-sensitive clients in edge networks consisting of multiple clients, access nodes (ANs), and servers. Clients request content through ANs acting as gateways, without observing AN states or the actions of other clients. We define the reward as the age of information reduction resulting from a client's selection of an AN, and formulate the problem as a non-stationary multi-armed bandit. In this decentralized and partially observable setting, the resulting reward process is history-dependent and coupled across clients, and exhibits both abrupt and gradual changes in expected rewards, rendering classical bandit-based approaches ineffective. To address these challenges, we propose the AGING BANDIT WITH ADAPTIVE RESET algorithm, which combines adaptive windowing with periodic monitoring to track evolving reward distributions. We establish theoretical performance guarantees showing that the proposed algorithm achieves near-optimal performance, and we validate the theoretical results through simulations.

Adaptive Requesting in Decentralized Edge Networks via Non-Stationary Bandits

TL;DR

This work tackles the problem of minimizing information freshness in decentralized edge networks where multiple clients route requests through edge gateways with limited coordination. By reframing the problem as a non-stationary Aging Bandit with history-dependent, correlated rewards, the authors introduce ABAR, a decentralized algorithm that combines adaptive windowing with an adaptive reset mechanism and periodic monitoring to track evolving reward dynamics. They provide a theoretical framework showing sublinear AoI regret and asymptotic optimality under mixed abrupt and gradual changes, extending drift-change analysis to decentralized, correlated settings. Simulations demonstrate that ABAR approaches the performance of a centralized oracle and outperforms classical decentralized bandit baselines across challenging scenarios, highlighting its practical impact for scalable, low-latency information delivery in edge networks.

Abstract

We study a decentralized collaborative requesting problem that aims to optimize the information freshness of time-sensitive clients in edge networks consisting of multiple clients, access nodes (ANs), and servers. Clients request content through ANs acting as gateways, without observing AN states or the actions of other clients. We define the reward as the age of information reduction resulting from a client's selection of an AN, and formulate the problem as a non-stationary multi-armed bandit. In this decentralized and partially observable setting, the resulting reward process is history-dependent and coupled across clients, and exhibits both abrupt and gradual changes in expected rewards, rendering classical bandit-based approaches ineffective. To address these challenges, we propose the AGING BANDIT WITH ADAPTIVE RESET algorithm, which combines adaptive windowing with periodic monitoring to track evolving reward distributions. We establish theoretical performance guarantees showing that the proposed algorithm achieves near-optimal performance, and we validate the theoretical results through simulations.
Paper Structure (68 sections, 7 theorems, 209 equations, 8 figures, 2 algorithms)

This paper contains 68 sections, 7 theorems, 209 equations, 8 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Notation
  4. Problem Formulation
  5. Network Model
  6. Age of Information
  7. Objectives and Policies
  8. From Optimization to Multi-Armed Bandits
  9. Challenges
  10. Myopic Reformulation and Reward Design
  11. Non-Stationary MAB and AoI Regrets
  12. The Aging Bandit with Adaptive Reset Algorithm
  13. Design Principles
  14. Implementation of the Design Principles
  15. Adaptive windowing with reset
  16. ...and 53 more sections

Key Result

Theorem 1

Suppose that Assumptions assu:MChangePoints, assu:monitoring_ChangePoints, assu:AbruptReset, assu:Drift-tolerant regret and assu:epsilon_upper_bound hold. Assume that $\mathcal{T}_c$ is a global change with constant $c_a$ (Definition Defn:GlobalChange). Let $\delta=\frac{1}{T^3}$ and choose paramete

Figures (8)

  • Figure 1: An example of decentralized an edge network.
  • Figure 2: Two illustrative service scenarios in a decentralized network. In the first, client $1$ requests content from server $p$ via AN $k$. Upon receiving the request, AN $k$ decides to command the server $p$ to generate a new packet. In the second, another client $j$ requests content from server $P$ via AN $K$, and the AN serves the request directly from its local cache.
  • Figure 3: Average AoI Performance Comparison
  • Figure 4: Cumulative AoI Regret Performance Comparison
  • Figure 5: relationship diagram if $\mu_{I, t} \leqslant \mu_{I, X_{T_m}+1} \leqslant \mu_{J, X_{T_m}+1}$.
  • ...and 3 more figures

Theorems & Definitions (32)

  • Definition 1
  • Remark 1
  • Remark 2
  • Definition 2: Gradual and Abrupt Changes
  • Definition 3: Change Points
  • Definition 4: Gradual Segment
  • Definition 5
  • Definition 6: Resets
  • Definition 7: Reset Times
  • Definition 8: Drift-Tolerant Regret, Definition 12 in JMLR
  • ...and 22 more