Lotus: Optimizing Disaggregated Transactions with Disaggregated Locks

TL;DR

Lotus tackles the MN RNIC bottleneck in disaggregated memory by moving locking out of data path onto CNs. Through application-aware lock sharding, a lock-first protocol, and a lock-rebuild-free recovery model, Lotus achieves significant throughput and latency improvements over state-of-the-art DM systems. Key innovations include a distributed lock table on CNs, two-level load balancing with shard-based routing, and a version-table cache to accelerate reads under MVCC. Experiments across KVS, TATP, SmallBank, and TPCC show up to 2.1x throughput gains and up to 49.4% latency reductions, with modest memory overhead, demonstrating practical impact for DM-based OLTP workloads.

Abstract

Disaggregated memory (DM) separates compute and memory resources, allowing flexible scaling to achieve high resource utilization. To ensure atomic and consistent data access on DM, distributed transaction systems have been adapted, where compute nodes (CNs) rely on one-sided RDMA operations to access remote data in memory nodes (MNs). However, we observe that in existing transaction systems, the RDMA network interface cards at MNs become a primary performance bottleneck. This bottleneck arises from the high volume of one-sided atomic operations used for locks, which hinders the system's ability to scale efficiently. To address this issue, this paper presents Lotus, a scalable distributed transaction system with lock disaggregation on DM. The key innovation of Lotus is to disaggregate locks from data and execute all locks on CNs, thus eliminating the bottleneck at MN RNICs. To achieve efficient lock management on CNs, Lotus employs an application-aware lock management mechanism that leverages the locality of the OLTP workloads to shard locks while maintaining load balance. To ensure consistent transaction processing with lock disaggregation, Lotus introduces a lock-first transaction protocol, which separates the locking phase as the first step in each read-write transaction execution. This protocol allows the system to determine the success of lock acquisitions early and proactively abort conflicting transactions, improving overall efficiency. To tolerate lock loss during CN failures, Lotus employs a lock-rebuild-free recovery mechanism that treats locks as ephemeral and avoids their reconstruction, ensuring lightweight recovery for CN failures. Experimental results demonstrate that Lotus improves transaction throughput by up to 2.1$\times$ and reduces latency by up to 49.4% compared to state-of-the-art transaction systems on DM.

Figures (23)

• Figure 1: The architecture of disaggregated memory.
• Figure 2: The throughput of Motor and FORD on SmallBank.
• Figure 3: The throughput of Motor and FORD on SmallBank after abandoning CAS.
• Figure 4: Decoupling data and locks on disaggregated memory.
• Figure 5: The overview of Lotus.