Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Herb.jl: A Unifying Program Synthesis Library

Tilman Hinnerichs, Reuben Gardos Reid, Jaap de Jong, Bart Swinkels, Pamela Wochner, Nicolae Filat, Tudor Magurescu, Issa Hanou, Sebastijan Dumancic

TL;DR

Herb.jl addresses fragmentation in program-synthesis research by introducing a modular, unifying Julia library that standardizes problem formulation into a grammar, specification, interpretation, constraints, and search components. Its design centers on interchangeable modules and uniform trees to enable easy reuse and fair comparisons across approaches, demonstrated through the Garden.jl ecosystem and HerbBenchmarks.jl. The paper showcases practical use cases, including reimplementing the Probe algorithm within Herb.jl and evaluating on standard benchmarks, to illustrate interoperability and reproducible workflows. This work enables researchers to remix existing synthesis techniques, compare methods on common grounds, and accelerate practical progress in program synthesis.

Abstract

Program synthesis -- the automatic generation of code given a specification -- is one of the most fundamental tasks in artificial intelligence (AI) and many programmers' dream. Numerous synthesizers have been developed to tackle program synthesis, manifesting different ideas to approach the exponentially growing program space. While numerous smart program synthesis tools exist, reusing and remixing previously developed methods is tedious and time-consuming. We propose Herb.jl, a unifying program synthesis library written in the Julia programming language, to address these issues. Since current methods rely on similar building blocks, we aim to modularize the underlying synthesis algorithm into communicating and fully extendable sub-compartments, allowing for straightforward reapplication of these modules. To demonstrate the benefits of using Herb.jl, we show three common use cases: 1. how to implement a simple problem and grammar, and how to solve it, 2. how to implement a previously developed synthesizer with just a few lines of code, and 3. how to run a synthesizer against a benchmark.

Herb.jl: A Unifying Program Synthesis Library

TL;DR

Herb.jl addresses fragmentation in program-synthesis research by introducing a modular, unifying Julia library that standardizes problem formulation into a grammar, specification, interpretation, constraints, and search components. Its design centers on interchangeable modules and uniform trees to enable easy reuse and fair comparisons across approaches, demonstrated through the Garden.jl ecosystem and HerbBenchmarks.jl. The paper showcases practical use cases, including reimplementing the Probe algorithm within Herb.jl and evaluating on standard benchmarks, to illustrate interoperability and reproducible workflows. This work enables researchers to remix existing synthesis techniques, compare methods on common grounds, and accelerate practical progress in program synthesis.

Abstract

Program synthesis -- the automatic generation of code given a specification -- is one of the most fundamental tasks in artificial intelligence (AI) and many programmers' dream. Numerous synthesizers have been developed to tackle program synthesis, manifesting different ideas to approach the exponentially growing program space. While numerous smart program synthesis tools exist, reusing and remixing previously developed methods is tedious and time-consuming. We propose Herb.jl, a unifying program synthesis library written in the Julia programming language, to address these issues. Since current methods rely on similar building blocks, we aim to modularize the underlying synthesis algorithm into communicating and fully extendable sub-compartments, allowing for straightforward reapplication of these modules. To demonstrate the benefits of using Herb.jl, we show three common use cases: 1. how to implement a simple problem and grammar, and how to solve it, 2. how to implement a previously developed synthesizer with just a few lines of code, and 3. how to run a synthesizer against a benchmark.

Paper Structure

This paper contains 25 sections, 3 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Problem 1: Synthesizer implementations are domain-specific.
  3. Problem 2: Synthesizers comprise the same building blocks that cannot be reused easily.
  4. Problem 3: Synthesizers are hard to compare due to varying engineering choices.
  5. Problem 4: Benchmarks are hard to reuse and rerun.
  6. Background
  7. Program Synthesis
  8. Overview and Design
  9. Components of Herb
  10. Core Modules
  11. Interchangeability of Modules
  12. Special Modules
  13. Design Choices
  14. Syntax and Semantics
  15. Uniform Trees
  16. ...and 10 more sections

Figures (2)

  • Figure 1: Dependency graph of packages in the ecosystem. Our system is hierarchical, i.e., all packages directly depend on all packages that are lower in the hierarchy. For example, the umbrella package directly depends on all packages. Due to its size, (all its sub-packages) do not depend on HerbBenchmarks.jl.
  • Figure 2: Example of a uniform tree, where the dashed boxes indicate uniform nodes, i.e., operators that have the same "shape": They have the same number of children with the same type.

Theorems & Definitions (3)

  • Example 1: The New Approach
  • Example 2: Getting Started
  • Example 3: Partial Program