Analysis and Evaluation of Using Microsecond-Latency Memory for In-Memory Indices and Caches in SSD-Based Key-Value Stores

TL;DR

This work analyzes the impact of microsecond-latency memory on SSD-based KV stores by modeling the interaction between software prefetching and IO. It shows that IO can significantly enhance latency-tolerance, enabling near-DRAM throughput even when in-memory indices and caches are placed on slower memory. The authors validate the theory with a microbenchmark and with modified KV stores (Aerospike, RocksDB, CacheLib) implemented on FPGA-based memory, achieving DRAM-like performance up to memory latencies of about 5 μs. The findings suggest microsecond-latency memory can be a cost-effective alternative to host DRAM, especially when combined with latency-hiding techniques and potential tiered memory configurations.

Abstract

When key-value (KV) stores use SSDs for storing a large number of items, oftentimes they also require large in-memory data structures including indices and caches to be traversed to reduce IOs. This paper considers offloading most of such data structures from the costly host DRAM to secondary memory whose latency is in the microsecond range, an order of magnitude longer than those of currently available DIMM-mounted or CXL memory devices. While emerging microsecond-latency memory is likely to cost much less than DRAM, it can significantly slow down SSD-based KV stores if naively employed. This paper analyzes and evaluates the impact of microsecond-level memory latency on the KV operation throughput. Our analysis finds that a well-known latency-hiding technique of software prefetching for long-latency memory from user-level threads is effective. The novelty of our analysis lies in modeling how the interplay between prefetching and IO affects performance, from which we derive an equation that well explains the throughput degradation due to long memory latency. The model tells us that the presence of IO significantly enhances the tolerance to memory latency, leading to a finding that SSD-based KV stores can be made latency-tolerant without devising new techniques for microsecond-latency memory. To confirm this, we design a microbenchmark as well as modify existing SSD-based KV stores so that they issue prefetches from user-level threads, and run them while placing most of in-memory data structures on FPGA-based memory with adjustable microsecond latency. The results demonstrate that their KV operation throughputs can be well explained by our model, and the modified KV stores achieve near-DRAM throughputs for up to a memory latency of 5 microseconds. This suggests the possibility that SSD-based KV stores can use microsecond-latency memory as a cost-effective alternative to the host DRAM.

Figures (18)

• Figure 1: (a) SSD-based KV stores often perform pointer-chasing on large in-memory indices and caches in addition to SSD access. We offload these in-memory data structures from the host DRAM to much slower, microsecond-latency memory. (b) The memory latency we study is in the microsecond range, an order of magnitude longer than those of currently available memory devices.
• Figure 2: Memory-only operation model
• Figure 3: Normalized throughput calculated from various models using example values in Table \ref{['tab:symbols']}
• Figure 4: Memory-only operation model using multiple threads with a large prefetch queue depth
• Figure 5: Memory-only operation model using multiple threads with a small prefetch queue depth