Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learned Graph Rewriting with Equality Saturation: A New Paradigm in Relational Query Rewrite and Beyond

George-Octavian Bărbulescu, Taiyi Wang, Zak Singh, Eiko Yoneki

TL;DR

This paper addresses the NP-hard problem of relational query rewrite by integrating equality saturation with graph reinforcement learning in a system named Aurora. By modeling equality saturation as a Markov decision process and employing a spatio-temporal RL agent that combines Graph Attention Networks with LSTMs, Aurora guides non-destructive e-graph expansion to construct many competitive query plans within practical memory and time budgets. The extraction of optimal plans is performed via ILP, with a cost model tied to end-to-end latency, enabling significant reductions in planning time and plan cost compared to state-of-the-art ES solvers and competitive performance against mainstream optimisers. The approach demonstrates strong runtime advantages on testbeds like Risinglight and PostgreSQL, and highlights RL’s pivotal role in navigating the combinatorial space of rewrite rules, offering a transferable paradigm for broader programming languages beyond SQL.

Abstract

Query rewrite systems perform graph substitutions using rewrite rules to generate optimal SQL query plans. Rewriting logical and physical relational query plans is proven to be an NP-hard sequential decision-making problem with a search space exponential in the number of rewrite rules. In this paper, we address the query rewrite problem by interleaving Equality Saturation and Graph Reinforcement Learning (RL). The proposed system, Aurora, rewrites relational queries by guiding Equality Saturation, a method from compiler literature to perform non-destructive graph rewriting, with a novel RL agent that embeds both the spatial structure of the query graph as well as the temporal dimension associated with the sequential construction of query plans. Our results show Graph Reinforcement Learning for non-destructive graph rewriting yields SQL plans orders of magnitude faster than existing equality saturation solvers, while also achieving competitive results against mainstream query optimisers.

Learned Graph Rewriting with Equality Saturation: A New Paradigm in Relational Query Rewrite and Beyond

TL;DR

This paper addresses the NP-hard problem of relational query rewrite by integrating equality saturation with graph reinforcement learning in a system named Aurora. By modeling equality saturation as a Markov decision process and employing a spatio-temporal RL agent that combines Graph Attention Networks with LSTMs, Aurora guides non-destructive e-graph expansion to construct many competitive query plans within practical memory and time budgets. The extraction of optimal plans is performed via ILP, with a cost model tied to end-to-end latency, enabling significant reductions in planning time and plan cost compared to state-of-the-art ES solvers and competitive performance against mainstream optimisers. The approach demonstrates strong runtime advantages on testbeds like Risinglight and PostgreSQL, and highlights RL’s pivotal role in navigating the combinatorial space of rewrite rules, offering a transferable paradigm for broader programming languages beyond SQL.

Abstract

Query rewrite systems perform graph substitutions using rewrite rules to generate optimal SQL query plans. Rewriting logical and physical relational query plans is proven to be an NP-hard sequential decision-making problem with a search space exponential in the number of rewrite rules. In this paper, we address the query rewrite problem by interleaving Equality Saturation and Graph Reinforcement Learning (RL). The proposed system, Aurora, rewrites relational queries by guiding Equality Saturation, a method from compiler literature to perform non-destructive graph rewriting, with a novel RL agent that embeds both the spatial structure of the query graph as well as the temporal dimension associated with the sequential construction of query plans. Our results show Graph Reinforcement Learning for non-destructive graph rewriting yields SQL plans orders of magnitude faster than existing equality saturation solvers, while also achieving competitive results against mainstream query optimisers.
Paper Structure (30 sections, 3 equations, 12 figures, 2 tables)

This paper contains 30 sections, 3 equations, 12 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Equality Graphs
  4. Equality Saturation
  5. Reinforcement Learning
  6. Problem Overview
  7. Challenges in Query Rewrite
  8. Challenges in Applying Equality Saturation
  9. Solution Overview
  10. Reinforcement Learning for Equality Saturation
  11. Equality Saturation as Markov Decision Process
  12. State
  13. Action & Transition Function
  14. Reward Signal
  15. Spatio-temporal Reinforcement Learning
  16. ...and 15 more sections

Figures (12)

  • Figure 1: E-graph construction. The e-graph undergoes rewrite $x*2 \rightarrow x<<1$. Figure adapted from willsey2021egg.
  • Figure 2: The original SQL plan (left) is optimised in a top-down fashion by creating a temporary view over the subquery (right).
  • Figure 3: A 6-step schematic workflow of Aurora
  • Figure 4: Example e-graph for RA expression $\pi(\sigma_{c1}(\sigma_{c2}(t)))$. By using the rewrite rule "$\sigma (c_1, \sigma (c_2, t) )\rightarrow \sigma( c_1 \land c_2, t)$", the query e-graph adds the equivalent plan $\pi(\sigma_{c1 \land c2}(t)))$. The circles represent e-node operators and the blue boxes represent e-classes
  • Figure 5: Example e-graph encoding. The encoding scheme covers both the nodes as well as the edges in the e-graph
  • ...and 7 more figures