Employ SmartNICs' Data Path Accelerators for Ordered Key-Value Stores

TL;DR

DPA-Store tackles the challenge of remote, ordered KV stores by leveraging the on-path Data Path Accelerators of the BlueField-3 SmartNIC to bypass OS overheads and enable stateless clients with range support. It uses a NIC-resident learned index to drive most of the lookup path, while maintaining a host-side replica for value storage and heavy maintenance tasks, coordinated through RCU-based stitching to keep the NIC traversal lock-free. The design achieves up to 33 MOPS for GET and 13 MOPS for RANGE, with INSERTs bottlenecked by host-to-DPA transfer bandwidth, and demonstrates competitive performance against state-of-the-art RDMA-based KV stores like ROLEX, while highlighting hardware improvements that could push performance even further. The work provides practical insights into memory access patterns on SmartNICs, cache strategies, and the trade-offs of learned indexes in NIC-resident data structures, offering a compelling direction for high-throughput, low-latency remote KV systems.

Abstract

Remote in-memory key-value (KV) stores serve as a cornerstone for diverse modern workloads, and high-speed range scans are frequently a requirement. However, current architectures rarely achieve a simultaneous balance of peak efficiency, architectural simplicity, and native support for ordered operations. Conventional host-centric frameworks are restricted by kernel-space network stacks and internal bus latencies. While hash-based alternatives that utilize OS-bypass or run natively on SmartNICs offer high throughput, they lack the data structures necessary for range queries. Distributed RDMA-based systems provide performance and range functionality but often depend on stateful clients, which introduces complexity in scaling and error handling. Alternatively, SmartNIC implementations that traverse trees located in host memory are hampered by high DMA round-trip latencies. This paper introduces a KV store that leverages the on-path Data Path Accelerators (DPAs) of the BlueField-3 SmartNIC to eliminate operating system overhead while facilitating stateless clients and range operations. These DPAs ingest network requests directly from NIC buffers to navigate a lock-free learned index residing in the accelerator's local memory. By deferring value retrieval from the host-side tree replica until the leaf level is reached, the design minimizes PCIe crossings. Write operations are staged in DPA memory and migrated in batches to the host, where structural maintenance is performed before being transactionally stitched back to the SmartNIC. Coupled with a NIC-resident read cache, the system achieves 33 million operations per second (MOPS) for point lookups and 13 MOPS for range queries. Our analysis demonstrates that this architecture matches or exceeds the performance of contemporary state-of-the-art solutions, while we identify hardware refinements that could further accelerate performance.

Figures (15)

• Figure 1: Access patterns in B-trees vs. learned trees.
• Figure 2: NVIDIA BlueField-3 architecture configured in NIC mode, allowing DMA access to host memory.
• Figure 3: DPA-Store architecture: (I) Request packets are assigned on-path to a DPA traverser thread, which (II) walks a NIC-side learned index. (III) Values are fetched per DMA using leaf-level models, and (IV) inserts are placed in leaf-level insert buffers. (V) The host performs structural updates that are (VI) transactionally stitched back to the NIC without interrupting traversers.
• Figure 4: Layout of NIC-side nodes and leaf DMA regions.
• Figure 5: DPA-side hot-entry cache.