Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Idiosyncrasies of Programmable Caching Engines

José Peixoto, Alexis Gonzalez, Janki Bhimani, Raju Rangaswami, Cláudia Brito, João Paulo, Ricardo Macedo

Abstract

Programmable caching engines like CacheLib are widely used in production systems to support diverse workloads in multi-tenant environments. CacheLib's design focuses on performance, portability, and configurability, allowing applications to inherit caching improvements with minimal implementation effort. However, its behavior under dynamic and evolving workloads remains largely unexplored. This paper presents an empirical study of CacheLib with multi-tenant settings under dynamic and volatile environments. Our evaluation across multiple CacheLib configurations reveals several limitations that hinder its effectiveness under such environments, including rigid configurations, limited runtime adaptability, lack of quality-of-service support and coordination, which lead to suboptimal performance, inefficient memory usage, and tenant starvation. Based on these findings, we outline future research directions to improve the adaptability, fairness, and programmability of future caching engines.

Idiosyncrasies of Programmable Caching Engines

Abstract

Programmable caching engines like CacheLib are widely used in production systems to support diverse workloads in multi-tenant environments. CacheLib's design focuses on performance, portability, and configurability, allowing applications to inherit caching improvements with minimal implementation effort. However, its behavior under dynamic and evolving workloads remains largely unexplored. This paper presents an empirical study of CacheLib with multi-tenant settings under dynamic and volatile environments. Our evaluation across multiple CacheLib configurations reveals several limitations that hinder its effectiveness under such environments, including rigid configurations, limited runtime adaptability, lack of quality-of-service support and coordination, which lead to suboptimal performance, inefficient memory usage, and tenant starvation. Based on these findings, we outline future research directions to improve the adaptability, fairness, and programmability of future caching engines.
Paper Structure (29 sections, 5 figures)

This paper contains 29 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Rigid configuration model
  3. Limited adaptability
  4. Lack of quality-of-service (QoS) support
  5. Lack of inter-instance coordination
  6. Background
  7. CacheLib overview
  8. Memory allocation and fragmentation management
  9. Multi-tenancy support
  10. Adaptive memory management
  11. CacheLib request workflow
  12. Foreground flow
  13. Background flow
  14. Usage in production environments
  15. Single instance, multiple tenants
  16. ...and 14 more sections

Figures (5)

  • Figure 1: CacheLib design overview (a) and typical applicability models in production (b-c).
  • Figure 2: Per-tenant performance comparison across CacheLib baseline, custom, and optimizer setups (columns). Each row depicts the throughput (kops/s) and allocated memory (GiB) over time of each tenant (T$_1$ to T$_4$). The left y-axis presents the throughput (line), while the right y-axis presents the memory allocated (filled area).
  • Figure 3: Overall throughput under different allocation policies. Workload-aware memory allocation substantially outperforms uniform policies.
  • Figure 4: Per-tenant performance comparison across CacheLib custom and optimizer setups under dynamic workloads. Each row depicts the throughput (kops/s) and allocated memory (GiB) over time of each tenant (T$_1$ to T$_4$). The left y-axis presents the throughput (line), while the right y-axis presents the memory allocated (filled area).
  • Figure 5: Overall throughput under dynamic workloads. Workload-aware reallocation outperforms static or delayed adaptation.