Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noise to the Rescue: Escaping Local Minima in Neurosymbolic Local Search

Alessandro Daniele, Emile van Krieken

TL;DR

This work recasts backpropagation in neurosymbolic learning as a discrete local search by applying Gödel logic to neural computations, revealing a direct link between gradient updates and variable flips in SAT solving. It introduces the Gödel Trick, a noise-based reparameterization that empowers exploration and escapes local minima, and demonstrates its effectiveness on SATLIB benchmarks and the Visual Sudoku task. The GT framework outperforms standard fuzzy and probabilistic logics on SAT, while matching state-of-the-art Visual Sudoku performance with substantially lower computational cost. Overall, the approach provides a scalable, differentiable pathway to integrate symbolic reasoning with neural models, with broad implications for neurosymbolic learning and reasoning under uncertainty.

Abstract

Deep learning has achieved remarkable success across various domains, largely thanks to the efficiency of backpropagation (BP). However, BP's reliance on differentiability poses challenges in neurosymbolic learning, where discrete computation is combined with neural models. We show that applying BP to Godel logic, which represents conjunction and disjunction as min and max, is equivalent to a local search algorithm for SAT solving, enabling the optimisation of discrete Boolean formulas without sacrificing differentiability. However, deterministic local search algorithms get stuck in local optima. Therefore, we propose the Godel Trick, which adds noise to the model's logits to escape local optima. We evaluate the Godel Trick on SATLIB, and demonstrate its ability to solve a broad range of SAT problems. Additionally, we apply it to neurosymbolic models and achieve state-of-the-art performance on Visual Sudoku, all while avoiding expensive probabilistic reasoning. These results highlight the Godel Trick's potential as a robust, scalable approach for integrating symbolic reasoning with neural architectures.

Noise to the Rescue: Escaping Local Minima in Neurosymbolic Local Search

TL;DR

This work recasts backpropagation in neurosymbolic learning as a discrete local search by applying Gödel logic to neural computations, revealing a direct link between gradient updates and variable flips in SAT solving. It introduces the Gödel Trick, a noise-based reparameterization that empowers exploration and escapes local minima, and demonstrates its effectiveness on SATLIB benchmarks and the Visual Sudoku task. The GT framework outperforms standard fuzzy and probabilistic logics on SAT, while matching state-of-the-art Visual Sudoku performance with substantially lower computational cost. Overall, the approach provides a scalable, differentiable pathway to integrate symbolic reasoning with neural models, with broad implications for neurosymbolic learning and reasoning under uncertainty.

Abstract

Deep learning has achieved remarkable success across various domains, largely thanks to the efficiency of backpropagation (BP). However, BP's reliance on differentiability poses challenges in neurosymbolic learning, where discrete computation is combined with neural models. We show that applying BP to Godel logic, which represents conjunction and disjunction as min and max, is equivalent to a local search algorithm for SAT solving, enabling the optimisation of discrete Boolean formulas without sacrificing differentiability. However, deterministic local search algorithms get stuck in local optima. Therefore, we propose the Godel Trick, which adds noise to the model's logits to escape local optima. We evaluate the Godel Trick on SATLIB, and demonstrate its ability to solve a broad range of SAT problems. Additionally, we apply it to neurosymbolic models and achieve state-of-the-art performance on Visual Sudoku, all while avoiding expensive probabilistic reasoning. These results highlight the Godel Trick's potential as a robust, scalable approach for integrating symbolic reasoning with neural architectures.

Paper Structure

This paper contains 46 sections, 14 theorems, 55 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Propositional logic
  4. Gödel logic semantics
  5. Related work
  6. Relation between Gödel and Boolean Logics
  7. Implicit interpretation
  8. Backpropagation on Gödel Logic
  9. Sparse gradients
  10. BP on CNFs formulas
  11. Comparison with common LSAs
  12. Local minima and cycles
  13. The Gödel Trick
  14. Gödel Trick as approximate probabilistic inference
  15. The choice of the noise distribution
  16. ...and 31 more sections

Key Result

Theorem 4.1

Consider a Gödel interpretation $\mathcal{G}$. Define the Boolean interpretation $\mathcal{B}$ obtained by setting $\mathcal{B}(p) = s(\mathcal{G}(p))$ for all propositional variables $p\in P$. Then for all formulas $\varphi$, we have $s(\mathcal{G}(\varphi))=\mathcal{B}(\varphi)$.

Figures (3)

  • Figure 1: Example of Gödel interpretation (left) and corresponding implicit interpretation (right) for the formula $\varphi = (A \lor B) \land \lnot C$. Each truth value in the implicit interpretation is either a $-1$ (false) or a $+1$ (true), and it can be computed via the sign function applied on the corresponding continue value in the Gödel interpretation.
  • Figure 2: Ratio of solved problems on the uf20-91 benchmark of SATLIB. Average of 100 runs over 50k epochs (granularity: 100 epochs).
  • Figure 3: A boolean interpretation $\mathcal{B}$ (top-left) for the formula $\varphi = \lnot (A \lor B) \land (B \lor C) \land (C \lor D)$ and three Gödel interpretations ($\mathcal{G}_1$, $\mathcal{G}_2$, and $\mathcal{G}_3$) that maps to $\mathcal{B}$. Each continuous interpretations have a different active path, leading to a different new version of $\mathcal{B}$.

Theorems & Definitions (34)

  • Theorem 4.1
  • Example 1
  • Definition 5.1: Path
  • Theorem 5.2
  • Definition 5.3: Active path
  • Theorem 5.4
  • Example 2
  • Theorem 5.5
  • Definition 5.6: Representation of a discrete interpretation
  • Definition 5.7: Candidate Path
  • ...and 24 more