Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compositional Program Generation for Few-Shot Systematic Generalization

Tim Klinger, Luke Liu, Soham Dan, Maxwell Crouse, Parikshit Ram, Alexander Gray

TL;DR

Addresses the challenge of systematic compositional generalization and data efficiency in neural models. Proposes the Compositional Program Generator (CPG), a neuro-symbolic architecture that uses modularity, composition, and abstraction via grammar rules to enable few-shot, productive generalization on sequence-to-sequence tasks, achieving perfect generalization on SCAN and COGS with just 14 and 22 examples. Demonstrates a framework where grammar-driven modules (copy and substitution) are learned per rule, with curricular training enabling incremental acquisition of semantics while preserving prior knowledge. The results show strong sample efficiency and interpretability, suggesting a scalable path toward applying modular, grammar-aware reasoning to real-world language tasks.

Abstract

Compositional generalization is a key ability of humans that enables us to learn new concepts from only a handful examples. Neural machine learning models, including the now ubiquitous Transformers, struggle to generalize in this way, and typically require thousands of examples of a concept during training in order to generalize meaningfully. This difference in ability between humans and artificial neural architectures, motivates this study on a neuro-symbolic architecture called the Compositional Program Generator (CPG). CPG has three key features: \textit{modularity}, \textit{composition}, and \textit{abstraction}, in the form of grammar rules, that enable it to generalize both systematically to new concepts in a few-shot manner, as well as productively by length on various sequence-to-sequence language tasks. For each input, CPG uses a grammar of the input language and a parser to generate a parse in which each grammar rule is assigned its own unique semantic module, a probabilistic copy or substitution program. Instances with the same parse are always processed with the same composed modules, while those with different parses may be processed with different modules. CPG learns parameters for the modules and is able to learn the semantics for new rules and types incrementally, without forgetting or retraining on rules it's already seen. It achieves perfect generalization on both the SCAN and COGS benchmarks using just 14 examples for SCAN and 22 examples for COGS -- state-of-the-art accuracy with a 1000x improvement in sample efficiency.

Compositional Program Generation for Few-Shot Systematic Generalization

TL;DR

Addresses the challenge of systematic compositional generalization and data efficiency in neural models. Proposes the Compositional Program Generator (CPG), a neuro-symbolic architecture that uses modularity, composition, and abstraction via grammar rules to enable few-shot, productive generalization on sequence-to-sequence tasks, achieving perfect generalization on SCAN and COGS with just 14 and 22 examples. Demonstrates a framework where grammar-driven modules (copy and substitution) are learned per rule, with curricular training enabling incremental acquisition of semantics while preserving prior knowledge. The results show strong sample efficiency and interpretability, suggesting a scalable path toward applying modular, grammar-aware reasoning to real-world language tasks.

Abstract

Compositional generalization is a key ability of humans that enables us to learn new concepts from only a handful examples. Neural machine learning models, including the now ubiquitous Transformers, struggle to generalize in this way, and typically require thousands of examples of a concept during training in order to generalize meaningfully. This difference in ability between humans and artificial neural architectures, motivates this study on a neuro-symbolic architecture called the Compositional Program Generator (CPG). CPG has three key features: \textit{modularity}, \textit{composition}, and \textit{abstraction}, in the form of grammar rules, that enable it to generalize both systematically to new concepts in a few-shot manner, as well as productively by length on various sequence-to-sequence language tasks. For each input, CPG uses a grammar of the input language and a parser to generate a parse in which each grammar rule is assigned its own unique semantic module, a probabilistic copy or substitution program. Instances with the same parse are always processed with the same composed modules, while those with different parses may be processed with different modules. CPG learns parameters for the modules and is able to learn the semantics for new rules and types incrementally, without forgetting or retraining on rules it's already seen. It achieves perfect generalization on both the SCAN and COGS benchmarks using just 14 examples for SCAN and 22 examples for COGS -- state-of-the-art accuracy with a 1000x improvement in sample efficiency.
Paper Structure (18 sections, 5 equations, 8 figures, 5 algorithms)

This paper contains 18 sections, 5 equations, 8 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. Approach
  3. Example: SCAN
  4. Example: COGS
  5. Model
  6. Dictionary distribution
  7. Module distributions
  8. Curricular Training
  9. Objective
  10. Curriculum
  11. Experiments
  12. Construction of the Few-shot datasets
  13. Results and Comparisons
  14. Incremental Re-Training
  15. Discussion
  16. ...and 3 more sections

Figures (8)

  • Figure 1: CPG architecture
  • Figure 2: An example of compositional program generation on SCAN. Copy modules are parameterized by distributions dependent only on the abstract rules. An example of a copy operation is shown at the bottom. Also see Algorithm \ref{['fig:copy']}.
  • Figure 3: An example of compositional program generation on COGS. Substitution modules are parameterized by distributions dependent only on the abstract rules. An example of a substitution operation mapping objects to slots is shown at the bottom. Also see Algorithm \ref{['fig:substitute']}.
  • Figure 4: The COGS dictionary. Non-verb mappings depend on the token being mapped. Verb mappings depend on the primitive type of the verb. This is why the dictionary requires both the primitive type and token.
  • Figure 5: Comparison of Approaches
  • ...and 3 more figures