Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Distributed Protocols with Query Rewrites [Technical Report]

David Chu, Rithvik Panchapakesan, Shadaj Laddad, Lucky Katahanas, Chris Liu, Kaushik Shivakumar, Natacha Crooks, Joseph M. Hellerstein, Heidi Howard

TL;DR

This work introduces rule-driven rewrites to scale distributed protocols by treating decoupling and partitioning as general, correct-by-construction optimizations applicable to any Dedalus-expressed protocol. By leveraging order-insensitivity, monotonicity, and data-dependency analyses (FDs/CDs), the authors devise preconditions and mechanisms for coordinating-free decoupling and partitioning, achieving substantial throughput gains across Voting, 2PC, and Paxos. The approach provides formal correctness arguments via local rewrites and linearizability, and demonstrates performance parity with state-of-the-art ad hoc rewrites in Paxos, while highlighting opportunities for automation. The results suggest a practical path toward automated optimizers for distributed protocols, with implications for scalable cloud systems and future research into adaptive, dataflow-inspired protocol design.

Abstract

Distributed protocols such as 2PC and Paxos lie at the core of many systems in the cloud, but standard implementations do not scale. New scalable distributed protocols are developed through careful analysis and rewrites, but this process is ad hoc and error-prone. This paper presents an approach for scaling any distributed protocol by applying rule-driven rewrites, borrowing from query optimization. Distributed protocol rewrites entail a new burden: reasoning about spatiotemporal correctness. We leverage order-insensitivity and data dependency analysis to systematically identify correct coordination-free scaling opportunities. We apply this analysis to create preconditions and mechanisms for coordination-free decoupling and partitioning, two fundamental vertical and horizontal scaling techniques. Manual rule-driven applications of decoupling and partitioning improve the throughput of 2PC by $5\times$ and Paxos by $3\times$, and match state-of-the-art throughput in recent work. These results point the way toward automated optimizers for distributed protocols based on correct-by-construction rewrite rules.

Optimizing Distributed Protocols with Query Rewrites [Technical Report]

TL;DR

This work introduces rule-driven rewrites to scale distributed protocols by treating decoupling and partitioning as general, correct-by-construction optimizations applicable to any Dedalus-expressed protocol. By leveraging order-insensitivity, monotonicity, and data-dependency analyses (FDs/CDs), the authors devise preconditions and mechanisms for coordinating-free decoupling and partitioning, achieving substantial throughput gains across Voting, 2PC, and Paxos. The approach provides formal correctness arguments via local rewrites and linearizability, and demonstrates performance parity with state-of-the-art ad hoc rewrites in Paxos, while highlighting opportunities for automation. The results suggest a practical path toward automated optimizers for distributed protocols, with implications for scalable cloud systems and future research into adaptive, dataflow-inspired protocol design.

Abstract

Distributed protocols such as 2PC and Paxos lie at the core of many systems in the cloud, but standard implementations do not scale. New scalable distributed protocols are developed through careful analysis and rewrites, but this process is ad hoc and error-prone. This paper presents an approach for scaling any distributed protocol by applying rule-driven rewrites, borrowing from query optimization. Distributed protocol rewrites entail a new burden: reasoning about spatiotemporal correctness. We leverage order-insensitivity and data dependency analysis to systematically identify correct coordination-free scaling opportunities. We apply this analysis to create preconditions and mechanisms for coordination-free decoupling and partitioning, two fundamental vertical and horizontal scaling techniques. Manual rule-driven applications of decoupling and partitioning improve the throughput of 2PC by and Paxos by , and match state-of-the-art throughput in recent work. These results point the way toward automated optimizers for distributed protocols based on correct-by-construction rewrite rules.
Paper Structure (62 sections, 1 theorem, 11 figures)

This paper contains 62 sections, 1 theorem, 11 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Running example
  4. Datalog$^\neg$
  5. Dedalus
  6. Further terminology
  7. Correctness
  8. Decoupling
  9. Mutually Independent Decoupling
  10. Monotonic Decoupling
  11. Functional Decoupling
  12. Partitioning
  13. Co-hashing
  14. Dependencies
  15. Partial partitioning
  16. ...and 47 more sections

Key Result

Lemma 1

Given facts $f_1,f_2$ of an input relation of $C_1$ in $I$, $\pi_T(f_1)$ is less than $\pi_T(f_2)$ if and only if for each node $p$, for each pair of corresponding facts $f_1',f_2'$ in $I'$ where $\pi_L(f_1') = \pi_L(f_2') = p$, we have $\pi_T(f_1') < \pi_T(f_2')$.

Figures (11)

  • Figure 1: Dataflow diagram for a verifiably-replicated KVS. Edges are labeled with corresponding line numbers; dashed edges represent asynchronous channels. Each gray bounding box represents a node; select nodes' dataflows are presented.
  • Figure 2: Running example after mutually independent decoupling.
  • Figure 3: Running example after monotonic decoupling.
  • Figure 4: Running example after functional decoupling.
  • Figure 5: Running example after partitioning with co-hashing.
  • ...and 6 more figures

Theorems & Definitions (4)

  • Definition 4.1
  • Definition A.1: Existence dependency
  • Definition A.2: No-change dependency
  • Lemma 1: Order consistency