Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Squirrel Parser: A Linear-Time PEG Packrat Parser Capable of Left Recursion and Optimal Error Recovery

Luke A. D. Hutchison

TL;DR

The paper presents the Squirrel parser, a PEG packrat parser that directly handles all forms of left recursion while delivering optimal error recovery with $O(n \cdot |G|)$ time and space. It derives a minimal algorithm from first principles using cycle detection, per-position state, iterative seed growth, and version-tagged memoization to compute the least fixed point of left recursive cycles. For error recovery, it establishes a two-phase design guided by foundational axioms and constraints, including phase isolation and bounded recovery, and proves completeness, correctness, and minimality. The results include a formal, implementable algorithm with a unified Match type system and phase tracking, a tight complexity bound, and extensive testing across 631 scenarios, demonstrating robust, scalable PEG parsing without grammar rewriting or annotation overhead.

Abstract

We present the squirrel parser, a PEG packrat parser that directly handles all forms of left recursion with optimal error recovery, while maintaining linear time complexity in the length of the input even in the presence of an arbitrary number of errors. Traditional approaches to handling left recursion in a recursive descent parser require grammar rewriting or complex algorithmic extensions. We derive a minimal algorithm from first principles: cycle detection via per-position state tracking and $O(1)$-per-LR-cycle communication from descendant to ancestor recursion frames, and fixed-point search via iterative expansion. For error recovery, we derived a set of four axioms and twelve constraints that must be imposed upon an optimal error recovery design to ensure completeness, correctness, optimality of performance, and intuitiveness of behavior. We utilized a constraint satisfaction mechanism to search the space of all possibilities, arriving at a provably optimal and robust error recovery strategy that maintains perfect performance linearity.

The Squirrel Parser: A Linear-Time PEG Packrat Parser Capable of Left Recursion and Optimal Error Recovery

TL;DR

The paper presents the Squirrel parser, a PEG packrat parser that directly handles all forms of left recursion while delivering optimal error recovery with time and space. It derives a minimal algorithm from first principles using cycle detection, per-position state, iterative seed growth, and version-tagged memoization to compute the least fixed point of left recursive cycles. For error recovery, it establishes a two-phase design guided by foundational axioms and constraints, including phase isolation and bounded recovery, and proves completeness, correctness, and minimality. The results include a formal, implementable algorithm with a unified Match type system and phase tracking, a tight complexity bound, and extensive testing across 631 scenarios, demonstrating robust, scalable PEG parsing without grammar rewriting or annotation overhead.

Abstract

We present the squirrel parser, a PEG packrat parser that directly handles all forms of left recursion with optimal error recovery, while maintaining linear time complexity in the length of the input even in the presence of an arbitrary number of errors. Traditional approaches to handling left recursion in a recursive descent parser require grammar rewriting or complex algorithmic extensions. We derive a minimal algorithm from first principles: cycle detection via per-position state tracking and -per-LR-cycle communication from descendant to ancestor recursion frames, and fixed-point search via iterative expansion. For error recovery, we derived a set of four axioms and twelve constraints that must be imposed upon an optimal error recovery design to ensure completeness, correctness, optimality of performance, and intuitiveness of behavior. We utilized a constraint satisfaction mechanism to search the space of all possibilities, arriving at a provably optimal and robust error recovery strategy that maintains perfect performance linearity.
Paper Structure (40 sections, 15 theorems, 4 equations, 7 figures, 2 tables)

This paper contains 40 sections, 15 theorems, 4 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. PEG Formalism
  3. Motivation: Shortcomings of Existing Approaches
  4. Left Recursion: Formal Derivation
  5. Problem Formalization
  6. Derivation from First Principles
  7. Algorithmic Components
  8. (1) Cycle Detection via State Tracking.
  9. (2) Iterative Expansion with Seed Growth.
  10. (3) Version-Tagged Memoization.
  11. Implementation
  12. Algorithm Mechanics
  13. Core Algorithm.
  14. PEG Clause Implementations.
  15. Functional Purity and Version Tagging
  16. ...and 25 more sections

Key Result

Theorem 1

For any left-recursive cycle at position $p$, there exists a fixed point: a terminal state where the cycle cannot match additional input.

Figures (7)

  • Figure 1: Implementation-level pseudocode for the entire squirrel parsing algorithm without error recovery (left), and several PEG clause types (right). The other PEG clause types can be easily implemented following the pattern shown, based on the PEG operator definitions.
  • Figure 2: The parse tree created by the squirrel parser for a range of different left recursion types and associativity.
  • Figure 3: Match result type system
  • Figure 4: Memoization entry with phase tracking
  • Figure 5: Cache validation with phase tracking
  • ...and 2 more figures

Theorems & Definitions (17)

  • Definition 1: Parser State
  • Definition 2: Left-Recursive Cycle
  • Theorem 1: Fixed Point Existence
  • Theorem 2: Bottom-Up Necessity
  • Theorem 3: Monotonic Length Increase
  • Theorem 4: Correctness
  • Theorem 5: Complexity Preservation
  • Theorem 6: Time Complexity
  • Theorem 7: Space Complexity
  • Theorem 8: LR as Fixed-Point Iteration
  • ...and 7 more