Revisiting Cache Freshness for Emerging Real-Time Applications

TL;DR

The paper tackles the limitation of TTL-based caching for real-time freshness in modern applications. It introduces a bounded-staleness model with costs $C_F$ and $C_S$ and shows that reacting to writes with updates or invalidates—batched over a window $T$—can reduce overhead compared to TTL policies. An adaptive per-object policy is derived to decide between updates and invalidates based on per-object costs $c_u$, $c_i$, $c_m$ and read/write probabilities, and practical realization uses Top-K and Count-min sketches to estimate $E[W]$. Simulations on multiple workloads demonstrate substantial gains in freshness-related throughput and latency, while the paper also outlines open challenges around reliable delivery, many-to-many dependencies, and eviction integration.

Abstract

Caching is widely used in industry to improve application performance by reducing data-access latency and taking the load off the backend infrastructure. TTLs have become the de-facto mechanism used to keep cached data reasonably fresh (i.e., not too out of date with the backend). However, the emergence of real-time applications requires tighter data freshness, which is impractical to achieve with TTLs. We discuss why this is the case, and propose a simple yet effective adaptive policy to achieve the desired freshness.

Figures (6)

• Figure 1: Lazy or cache-aside caches; the predominant way in which in-memory caches are deployed today. In such caches, data freshness is not guaranteed since writes bypass the cache. Servicing a miss can either be initiated by the cache or by the application.
• Figure 2: Effect of decreasing staleness bound on normalized staleness cost ($C_S'$, §\ref{['sec:motivation']}). The x-axis is in log scale.
• Figure 3: Effect of decreasing staleness bound on normalized freshness cost ($C_F'$, §\ref{['sec:motivation']}). Both the x and the y axis are in log scale.
• Figure 4: System Overview. Depending on the workload pattern, the policy dynamically decides between invalidation and updates. Invalidates or updates are buffered at the data store and batched over $T$.
• Figure 5: Comparison to baselines. Adpt. denotes our proposed adaptive policy. Adpt. + C.S. denotes data store knowing which keys are present in the cache (C.S.). Opt. denotes the optimal policy. The left x-axis is in the log scale. The blue bar indicates $C_F'$ (left axis, in $\times$), the green bar indicates $C_S'$ (right axis, in $\%$), defined in §\ref{['sec:background-ttl']}. The y-axis is in the log scale.