Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fresh Caching of Dynamic Contents using Restless Multi-armed Bandits

Ankita Koley, Chandramani Singh

TL;DR

This work addresses dynamic content caching under updating contents by modeling the problem as a continuous-time RMAB with partial observability. By reformulating per-content dynamics as a semi-Markov decision process and proving indexability, the authors derive closed-form Whittle indices and implement a Whittle-index policy that caches the $M$ contents with the highest indices. The approach mitigates the curse of dimensionality and yields near-optimal performance compared to the relaxed RMAB benchmark, while outperforming prior cache policies in simulations. The results offer a principled, scalable framework for dynamic caching where content freshness and fetch costs must be balanced under hard cache constraints.

Abstract

We consider a dynamic content caching problem wherein the contents get updated at a central server, and local copies of a subset of contents are cached at a local cache associated with a Base station (BS). When a content request arrives, based on whether the content is in the local cache, the BS can decide whether to fetch the content from the central server or serve the cached version from the local cache. Fetching a content incurs a fixed fetching cost, and serving the cached version incurs an ageing cost proportional to the age-of-version (AoV) of the content. The BS has only partial information regarding AoVs of the contents. We formulate an optimal content fetching and caching problem to minimize the average cost subject to cache capacity constraints. The problem suffers from the curse of dimensionality and is provably hard to solve. We formulate this problem as a continuous time restless multi-armed bandit process (RMAB), where a single content problem of the corresponding RMAB is a partially observable Markov decision process. We reformulate the single content problem as a semi-Markov decision process, prove indexability, and provide a Whittle index based solution to this problem. Finally, we compare the performance with recent work and show that our proposed policy is optimal via simulations.

Fresh Caching of Dynamic Contents using Restless Multi-armed Bandits

TL;DR

This work addresses dynamic content caching under updating contents by modeling the problem as a continuous-time RMAB with partial observability. By reformulating per-content dynamics as a semi-Markov decision process and proving indexability, the authors derive closed-form Whittle indices and implement a Whittle-index policy that caches the contents with the highest indices. The approach mitigates the curse of dimensionality and yields near-optimal performance compared to the relaxed RMAB benchmark, while outperforming prior cache policies in simulations. The results offer a principled, scalable framework for dynamic caching where content freshness and fetch costs must be balanced under hard cache constraints.

Abstract

We consider a dynamic content caching problem wherein the contents get updated at a central server, and local copies of a subset of contents are cached at a local cache associated with a Base station (BS). When a content request arrives, based on whether the content is in the local cache, the BS can decide whether to fetch the content from the central server or serve the cached version from the local cache. Fetching a content incurs a fixed fetching cost, and serving the cached version incurs an ageing cost proportional to the age-of-version (AoV) of the content. The BS has only partial information regarding AoVs of the contents. We formulate an optimal content fetching and caching problem to minimize the average cost subject to cache capacity constraints. The problem suffers from the curse of dimensionality and is provably hard to solve. We formulate this problem as a continuous time restless multi-armed bandit process (RMAB), where a single content problem of the corresponding RMAB is a partially observable Markov decision process. We reformulate the single content problem as a semi-Markov decision process, prove indexability, and provide a Whittle index based solution to this problem. Finally, we compare the performance with recent work and show that our proposed policy is optimal via simulations.
Paper Structure (19 sections, 4 theorems, 34 equations, 5 figures, 2 tables, 1 algorithm)

This paper contains 19 sections, 4 theorems, 34 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Our Contribution
  4. System Model
  5. The Optimal Content Fetching and Caching Problem
  6. Single Content Problem
  7. Indexability
  8. Whittle Index policy for Multi-content problem
  9. Whittle Index Policy for Multi-content having Different Sizes
  10. Performance Analysis
  11. Comparison
  12. Effect of update rate ($\lambda$) on average cost
  13. Effect of fetching cost ($c_f$) on average cost and number of fetches during cache hits
  14. Conclusion
  15. Proof of Lemma 1
  16. ...and 4 more sections

Key Result

Lemma 1

Figures (5)

  • Figure 1: Fresh caching of dynamic content
  • Figure 2: Optimal policy structure with respect to $C_h$ for the state $(\tau,1,1)$ and $(\tau,1,0)$ where $\tau^{\ast},\tau^0$ and $I$ are as in Theorem \ref{['main_theorem_for_different_C_h']}.
  • Figure 3: Comparison of average cost between Whittle Index based policy and the policy by Abolhassani et al.9488731
  • Figure 4: Effect of update rate and cache size on average ageing cost and fetching cost
  • Figure 5: Effect of fetching cost $(c_f)$ on average cost

Theorems & Definitions (7)

  • Remark 1
  • Lemma 1
  • Theorem 2
  • Remark 2
  • Lemma 3
  • Remark 3
  • Theorem 4