Outback: Fast and Communication-efficient Index for Key-Value Store on Disaggregated Memory

TL;DR

Outback tackles the challenge of achieving fast, low-CPU-overhead KV lookups over disaggregated memory by decoupling its index using dynamic minimal perfect hashing. The compute side stores a compact, compute-heavy bucket locator and seeds, while the memory side stores the data-addressed DMPH buckets, enabling one-round-trip RDMA RPC with minimal memory-node computation. Through extensive experiments on YCSB and real-world SOSD datasets, Outback delivers 1.11–5.03x higher throughput than state-of-the-art one-sided and two-sided RDMA baselines, with modest memory overhead on compute nodes. This approach promises significant practical impact for scalable, low-latency KV stores in data centers and cloud environments with disaggregated memory resources.

Abstract

Disaggregated memory systems achieve resource utilization efficiency and system scalability by distributing computation and memory resources into distinct pools of nodes. RDMA is an attractive solution to support high-throughput communication between different disaggregated resource pools. However, existing RDMA solutions face a dilemma: one-sided RDMA completely bypasses computation at memory nodes, but its communication takes multiple round trips; two-sided RDMA achieves one-round-trip communication but requires non-trivial computation for index lookups at memory nodes, which violates the principle of disaggregated memory. This work presents Outback, a novel indexing solution for key-value stores with a one-round-trip RDMA-based network that does not incur computation-heavy tasks at memory nodes. Outback is the first to utilize dynamic minimal perfect hashing and separates its index into two components: one memory-efficient and compute-heavy component at compute nodes and the other memory-heavy and compute-efficient component at memory nodes. We implement a prototype of Outback and evaluate its performance in a public cloud. The experimental results show that Outback achieves higher throughput than both the state-of-the-art one-sided RDMA and two-sided RDMA-based in-memory KVS by 1.06-5.03x, due to the unique strength of applying a separated perfect hashing index.

Figures (17)

• Figure 1: Examples of two types of RDMA systems.
• Figure 2: Ludo hashing.
• Figure 3: Observations from the microbenchmarks.
• Figure 4: Outback overview
• Figure 5: The data layout in a DMPH bucket.