Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Enumeration of Recursive Plans in Transformation-based Query Optimizers

Amela Fejza, Pierre Genevès, Nabil Layaïda

TL;DR

This work advances query optimization by extending the transformation-based RLQDAG framework to capture and manipulate recursive queries. It introduces annotated equivalence nodes and a formal syntax/semantics that support grouped transformations and incremental annotation updates, enabling efficient exploration of large recursive plan spaces. The RLQDAG rewrite rules and expansion algorithm unify recursive and non-recursive rewrites, and empirical results show substantial runtime gains over state-of-the-art recursive plan enumerators, enhancing practical feasibility for complex recursive workloads. Overall, RLQDAG delivers a scalable, principled approach to enumerate and evaluate recursive plans, promising significant impact for RA-based optimizers and recursive query processing in databases.

Abstract

Query optimizers built on the transformation-based Volcano/Cascades framework are used in many database systems. Transformations proposed earlier on the logical query dag (LQDAG) data structure, which is key in such a framework, focus only on recursion-free queries. In this paper, we propose the recursive logical query dag (RLQDAG) which extends the LQDAG with the ability to capture and transform recursive queries, leveraging recent developments in recursive relational algebra. Specifically, this extension includes: (i) the ability of capturing and transforming sets of recursive relational terms thanks to (ii) annotated equivalence nodes used for guiding transformations that are more complex in the presence of recursion; and (iii) RLQDAG rewrite rules that transform sets of subterms in a grouped manner, instead of transforming individual terms in a sequential manner; and that (iv) incrementally update the necessary annotations. Core concepts of the RLQDAG are formalized using a syntax and formal semantics with a particular focus on subterm sharing and recursion. The result is a clean generalization of the LQDAG transformation-based approach, enabling more efficient explorations of plan spaces for recursive queries. An implementation of the proposed approach shows significant performance gains compared to the state-of-the-art.

Efficient Enumeration of Recursive Plans in Transformation-based Query Optimizers

TL;DR

This work advances query optimization by extending the transformation-based RLQDAG framework to capture and manipulate recursive queries. It introduces annotated equivalence nodes and a formal syntax/semantics that support grouped transformations and incremental annotation updates, enabling efficient exploration of large recursive plan spaces. The RLQDAG rewrite rules and expansion algorithm unify recursive and non-recursive rewrites, and empirical results show substantial runtime gains over state-of-the-art recursive plan enumerators, enhancing practical feasibility for complex recursive workloads. Overall, RLQDAG delivers a scalable, principled approach to enumerate and evaluate recursive plans, promising significant impact for RA-based optimizers and recursive query processing in databases.

Abstract

Query optimizers built on the transformation-based Volcano/Cascades framework are used in many database systems. Transformations proposed earlier on the logical query dag (LQDAG) data structure, which is key in such a framework, focus only on recursion-free queries. In this paper, we propose the recursive logical query dag (RLQDAG) which extends the LQDAG with the ability to capture and transform recursive queries, leveraging recent developments in recursive relational algebra. Specifically, this extension includes: (i) the ability of capturing and transforming sets of recursive relational terms thanks to (ii) annotated equivalence nodes used for guiding transformations that are more complex in the presence of recursion; and (iii) RLQDAG rewrite rules that transform sets of subterms in a grouped manner, instead of transforming individual terms in a sequential manner; and that (iv) incrementally update the necessary annotations. Core concepts of the RLQDAG are formalized using a syntax and formal semantics with a particular focus on subterm sharing and recursion. The result is a clean generalization of the LQDAG transformation-based approach, enabling more efficient explorations of plan spaces for recursive queries. An implementation of the proposed approach shows significant performance gains compared to the state-of-the-art.
Paper Structure (35 sections, 3 theorems, 4 equations, 26 figures, 1 table)

This paper contains 35 sections, 3 theorems, 4 equations, 26 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Plan enumeration for non-recursive queries
  4. The logical query dag (LQDAG)
  5. The RLQDAG
  6. Syntax
  7. Semantics of RLQDAG terms
  8. Recursive terms and rule applicability
  9. Preliminary definitions for RLQDAG
  10. Notion of destabilizer in RLQDAG
  11. Notion of rigidity in RLQDAG
  12. Annotated equivalence node
  13. Notion of consistency
  14. RLQDAG Transformations
  15. RLQDAG rewrite rules
  16. ...and 20 more sections

Key Result

Proposition 1

Let $\hm{\bigr[} \alpha \hm{\bigr]}$ be a consistent RLQDAG, and $\alpha' = \texttt{expand}(\hm{\bigr[} \alpha \hm{\bigr]})$, then we have $S_{\gamma}\llbracket {\hm{\bigr[} \alpha \hm{\bigr]}} \rrbracket_{\emptyset} \subseteq S_{\gamma}\llbracket \alpha' \rrbracket_{\emptyset}$ and $\alpha'$ is con

Figures (26)

  • Figure 1: Sample LQDAG (in green) with expansion (in blue).
  • Figure 2: Syntax of RLQDAG terms.
  • Figure 3: Structure of recursive terms in RLQDAG.
  • Figure 4: Formal semantics of RLQDAG terms, with one interpretation function for each syntactic construct of Fig. \ref{['fig:syntax_muLQDAG']}.
  • Figure 5: Well-formed.
  • ...and 21 more figures

Theorems & Definitions (9)

  • Definition 1: Well-formedness
  • Definition 2: Unfolding
  • Definition 3
  • Definition 4: Rigidity
  • Definition 5
  • Definition 6: Consistency
  • Proposition 1: Correctness
  • Proposition 2: Completeness properties
  • Proposition 1: Correctness