Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Online Gradient-Based Caching Policy with Logarithmic Complexity and Regret Guarantees

Damiano Carra, Giovanni Neglia

TL;DR

The paper tackles robust caching under arbitrary traffic by introducing OGB, an integral online gradient-based caching policy that achieves $O(\log N)$ amortized per-request complexity while maintaining sub-linear regret guarantees. It couples a gradient-driven probability update with a coordinated sampling step to realize a cache of fixed size, supporting batched and fractional extensions. Experimental results on large-scale traces demonstrate that gradient-based caching can outperform traditional policies and approach the optimal static allocation in several scenarios, validating the practical value of regret guarantees. The work opens avenues for extensions to heterogeneous item sizes and to broader online convex optimization problems in caching and related systems.

Abstract

Commonly used caching policies, such as LRU (Least Recently Used) or LFU (Least Frequently Used), exhibit optimal performance only under specific traffic patterns. Even advanced machine learning-based methods, which detect patterns in historical request data, struggle when future requests deviate from past trends. Recently, a new class of policies has emerged that are robust to varying traffic patterns. These algorithms address an online optimization problem, enabling continuous adaptation to the context. They offer theoretical guarantees on the regret metric, which measures the performance gap between the online policy and the optimal static cache allocation in hindsight. However, the high computational complexity of these solutions hinders their practical adoption. In this study, we introduce a new variant of the gradient-based online caching policy that achieves groundbreaking logarithmic computational complexity relative to catalog size, while also providing regret guarantees. This advancement allows us to test the policy on large-scale, real-world traces featuring millions of requests and items - a significant achievement, as such scales have been beyond the reach of existing policies with regret guarantees. To the best of our knowledge, our experimental results demonstrate for the first time that the regret guarantees of gradient-based caching policies offer substantial benefits in practical scenarios.

An Online Gradient-Based Caching Policy with Logarithmic Complexity and Regret Guarantees

TL;DR

The paper tackles robust caching under arbitrary traffic by introducing OGB, an integral online gradient-based caching policy that achieves amortized per-request complexity while maintaining sub-linear regret guarantees. It couples a gradient-driven probability update with a coordinated sampling step to realize a cache of fixed size, supporting batched and fractional extensions. Experimental results on large-scale traces demonstrate that gradient-based caching can outperform traditional policies and approach the optimal static allocation in several scenarios, validating the practical value of regret guarantees. The work opens avenues for extensions to heterogeneous item sizes and to broader online convex optimization problems in caching and related systems.

Abstract

Commonly used caching policies, such as LRU (Least Recently Used) or LFU (Least Frequently Used), exhibit optimal performance only under specific traffic patterns. Even advanced machine learning-based methods, which detect patterns in historical request data, struggle when future requests deviate from past trends. Recently, a new class of policies has emerged that are robust to varying traffic patterns. These algorithms address an online optimization problem, enabling continuous adaptation to the context. They offer theoretical guarantees on the regret metric, which measures the performance gap between the online policy and the optimal static cache allocation in hindsight. However, the high computational complexity of these solutions hinders their practical adoption. In this study, we introduce a new variant of the gradient-based online caching policy that achieves groundbreaking logarithmic computational complexity relative to catalog size, while also providing regret guarantees. This advancement allows us to test the policy on large-scale, real-world traces featuring millions of requests and items - a significant achievement, as such scales have been beyond the reach of existing policies with regret guarantees. To the best of our knowledge, our experimental results demonstrate for the first time that the regret guarantees of gradient-based caching policies offer substantial benefits in practical scenarios.
Paper Structure (22 sections, 1 theorem, 12 equations, 12 figures, 1 table, 3 algorithms)

This paper contains 22 sections, 1 theorem, 12 equations, 12 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background and motivation
  3. Online Gradient Based caching policy
  4. Motivation
  5. Solution overview
  6. Updating the Probabilities
  7. Improving Projection: Main Idea
  8. Improving Projection: The Algorithm
  9. Updating the Sample
  10. Coordination of Samples
  11. Item selection: The algorithm
  12. Fractional Caching
  13. Experiments
  14. Traces
  15. Results
  16. ...and 7 more sections

Key Result

theorem 1

The policy OGB in Algorithm alg:overall_scheme with $\eta = \sqrt{\frac{C\left(1- \frac{C}{N}\right)}{T B}}$ has regret upperbounded by $\sqrt{C\left(1- \frac{C}{N}\right) TB}$.

Figures (12)

  • Figure 1: Trace length ($T$) and catalog size ($N$) used in no-regret caching papers ($\blacklozenge$), and used commonly for evaluating caching policies ($\bigstar$).
  • Figure 2: Adversarial trace. While recency or frequency based policies are not able to cope with adversarial patterns, online gradient based policies show a close-to-optimal performance.
  • Figure 3: Real-world short traces: sensitivity of OGB$_{\text{cl}}$ and FTPL. In OGB$_{\text{cl}}$, the parameter $\eta$ is the learning rate, while in FTPL the parameter $\zeta$ is the noise added to the LFU counters.
  • Figure 4: Real-world long traces: sensitivity of OGB$_{\text{cl}}$ and FTPL. The initial noise added by FTPL heavily influences the performance.
  • Figure 5: OGB scheme
  • ...and 7 more figures

Theorems & Definitions (2)

  • theorem 1
  • proof