Neural-Symbolic Recursive Machine for Systematic Generalization

TL;DR

The paper tackles systematic generalization in sequence-to-sequence tasks by learning a Grounded Symbol System (GSS) that grounds perception, syntax, and semantics directly from data. It introduces the Neural-Symbolic Recursive Machine (NSR), a modular architecture with neural perception, dependency parsing, and program induction, trained via a deduction-abduction algorithm that provides pseudo supervision for non-differentiable components. NSR demonstrates state-of-the-art generalization across SCAN, PCFG, Hint, and a compositional machine translation task, achieving 100% generalization on SCAN and PCFG and substantial gains on Hint and MT, with less reliance on domain-specific curricula. While offering strong transferability and interpretability through the GSS, the approach acknowledges limitations related to noisy concepts expanding the symbol space and the deterministic nature of the learned programs, outlining avenues for future improvement.

Abstract

Current learning models often struggle with human-like systematic generalization, particularly in learning compositional rules from limited data and extrapolating them to novel combinations. We introduce the Neural-Symbolic Recursive Machine (NSR), whose core is a Grounded Symbol System (GSS), allowing for the emergence of combinatorial syntax and semantics directly from training data. The NSR employs a modular design that integrates neural perception, syntactic parsing, and semantic reasoning. These components are synergistically trained through a novel deduction-abduction algorithm. Our findings demonstrate that NSR's design, imbued with the inductive biases of equivariance and compositionality, grants it the expressiveness to adeptly handle diverse sequence-to-sequence tasks and achieve unparalleled systematic generalization. We evaluate NSR's efficacy across four challenging benchmarks designed to probe systematic generalization capabilities: SCAN for semantic parsing, PCFG for string manipulation, HINT for arithmetic reasoning, and a compositional machine translation task. The results affirm NSR's superiority over contemporary neural and hybrid models in terms of generalization and transferability.

Figures (9)

• Figure 1: Illustrative examples of gss demonstrating human-like reasoning processes. (a) SCAN: each node encapsulates a triplet (word, symbol, action sequence). (b) PCFG: nodes consist of triplets (word, symbol, letter list). (c) HINT: nodes contain triplets (image, symbol, value). Symbols are denoted by their indices.
• Figure 2: Illustration of the inference and learning pipeline in nsr.
• Figure 3: Illustration of abduction in Hint over perception, syntax, and semantics. Elements modified during abduction are emphasized in red.
• Figure 4: (a) Syntactic similarity amongst input words in nsr trained on the Length split in SCAN. The similarity between word $i$ and word $j$ is quantified by the percentage of test samples where substituting $i$ with $j$, or vice versa, retains the dependencies as predicted by the dependency parser. (b) Induced programs for input words using nsr. Here, $x$ and $y$ represent the inputs, $\varnothing$ signifies empty inputs, cons appends an item to the beginning of a list, car retrieves the first item of a list, and + amalgamates two lists.
• Figure A1: Applying the transition-based dependency parser to an example of Hint. It is similar for SCAN and PCFG.

Theorems & Definitions (5)

• Theorem 3.1
• Definition 3.1: Equivariance
• Definition 3.2: Compositionality
• Lemma C.1
• Lemma C.2