Reliable Replication Protocols on SmartNICs
M. R. Siavash Katebzadeh, Antonios Katsarakis, Boris Grot
TL;DR
The paper addresses the challenge of providing strong consistency in replicated KV-stores without sacrificing throughput and latency. It proposes Chaapar, a hardware-accelerated replication system that offloads protocol logic to SmartNICs, featuring a NIC-based consistency controller, dual-path caching, and a datastore interface to integrate with existing datastores and cloud storage. By analyzing CPU overheads across state-of-the-art replication protocols and leveraging RDMA-enabled NICs, the authors argue for reduced host CPU load and improved end-to-end latency. An initial implementation on BlueField-3 demonstrates the feasibility of offloading key replication tasks, while acknowledging practical constraints and outlining future work to support broader protocols and production deployment.
Abstract
Today's datacenter applications rely on datastores that are required to provide high availability, consistency, and performance. To achieve high availability, these datastores replicate data across several nodes. Such replication is managed through a reliable protocol designed to keep the replicas consistent using a consistency model, even in the presence of faults. For several applications, strong consistency models are favored over weaker consistency models, as the former guarantee a more intuitive behavior for clients. Furthermore, to meet the demands of high online traffic, datastores must offer high throughput and low latency. However, delivering both strong consistency and high performance simultaneously can be challenging. Reliable replication protocols typically require multiple rounds of communication over the network stack, which introduces latency and increases the load on network resources. Moreover, these protocols consume considerable CPU resources, which impacts the overall performance of applications, especially in high-throughput environments. In this work, we aim to design a hardware-accelerated system for replication protocols to address these challenges. We approach offloading the replication protocol onto SmartNICs, which are specialized network interface cards that can be programmed to implement custom logic directly on the NIC. By doing so, we aim to enhance performance while preserving strong consistency, all while saving valuable CPU cycles that can be used for applications' logic.