Utility-driven Optimization of TTL Cache Hierarchies under Network Delays

TL;DR

The object decoupling effect is leveraged to formulate the non-linear utility maximization problem for TTL cache hierarchies in terms of the exact object hit probability under random network delays and iteratively solve the utility maximization problem to find the optimal per-object TTLs.

Abstract

We optimize hierarchies of Time-to-Live (TTL) caches under random network delays. A TTL cache assigns individual eviction timers to cached objects that are usually refreshed upon a hit where upon a miss the object requires a random time to be fetched from a parent cache. Due to their object decoupling property, TTL caches are of particular interest since the optimization of a per-object utility enables service differentiation. However, state-of-the-art exact TTL cache optimization does not extend beyond single TTL caches, especially under network delays. In this paper, we leverage the object decoupling effect to formulate the non-linear utility maximization problem for TTL cache hierarchies in terms of the exact object hit probability under random network delays. We iteratively solve the utility maximization problem to find the optimal per-object TTLs. Further, we show that the exact model suffers from tractability issues for large hierarchies and propose a machine learning approach to estimate the optimal TTL values for large systems. Finally, we provide numerical and data center trace-based evaluations for both methods showing the significant offloading improvement due to TTL optimization considering the network delays.

Figures (7)

• Figure 1: Impact of delays on the TTL caching performance: The aggregate utilityof a 3-cache binary tree for increasing network delays and a fixed request rate at the leaves. The TTL caches are jointly optimized under the state-of-the-art zero network delay assumption. The performance of the TTL caching system deteriorates with network delays below delay-agnostic hierarchies built upon LRU, FIFO or random caching.
• Figure 2: Two level caching hierarchy tree consisting of $n_l$ leaves. The random variable $d_i$ encodes the delay for cache $i$ to download the content from its parent. The server contains all objects. The request rate of object $i$ at leaf $j$ is denoted $\lambda_{I,ij}$.
• Figure 3: The MAP of a single cache object with parameters $\lambda$, $\mu$ and $\mu_F$ for the inter-request time, TTL and delay distributions, respectively. States "0", "1" and "F" indicate that the object is in the cache, the object is out of the cache and the object is being fetched, respectively. The TTL and the delay are exponentially distributed. (a) represents a Poisson request process, while (b) represents an Erlang-2 request process. State pairs $X_a$ and $X_b$ correspond to the object during each of the two Erlang-2 request phases.
• Figure 4: Single cache results: TTL optimization under random network delay vs. the idealized zero-delay assumption where $\rho_d$ denotes the ratio of the expected network delay to the expected inter-request time of the hottest object. (a) Significant Improvement of origin offloading, i.e. traffic served by the cache, due to optimizing under delays (OPT$|_{\text{delay}}$) vs. idealization (OPT$|_{\text{ideal}}$) increases with the delay. (b) The optimal TTL values are larger for OPT$|_{\text{delay}}$ especially for hottest objects. Object IDs sorted in descending arrival rate.
• Figure 5: Two-level binary tree: (a) Aggregate utility of OPT$|_{\text{delay}}$ vs.OPT$|_{\text{ideal}}$ vs. LRU hierarchy. (b) Utility loss under strict cache capacity constraint with respect to the optimal utility without strict capacity restriction \ref{['eq:Main_opt_prob_avg_occupancy_constraint']}. TTL$_{\min}$ has a $10\%$ deviation due to the cache storage violation while the heuristic TTL$_{\min,\text{extnd}}$ approaches the optimal utility.

Theorems & Definitions (1)

• Theorem 1