Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cellular automata, many-valued logic, and deep neural networks

Yani Zhang, Helmut Bölcskei

TL;DR

The paper addresses the problem of uncovering the logical rules governing cellular automata and extracting them from neural network traces. It develops a theoretical bridge between CA and Łukasiewicz MV logic, leveraging the McNaughton theorem to show that CA transition functions correspond to MV term functions, which deep ReLU networks can realize. By extending to divisible MV algebras (DMV), it accommodates rational coefficients in piecewise-linear interpolations and proves a universal NN representation for CA transitions, further embedding these into RNNs to realize CA dynamics. It also provides an algorithm to read out DMV formulas from trained networks, enabling identification of CA logic from evolution data, with a software implementation and explicit examples including retrieval of MV terms like τ = x_{-1} ⊕ x_0 ⊕ x_1. This work advances interpretable rule extraction from data-generated CA dynamics and links symbolic logic with neural architectures for dynamical systems.

Abstract

We develop a theory characterizing the fundamental capability of deep neural networks to learn, from evolution traces, the logical rules governing the behavior of cellular automata (CA). This is accomplished by first establishing a novel connection between CA and Lukasiewicz propositional logic. While binary CA have been known for decades to essentially perform operations in Boolean logic, no such relationship exists for general CA. We demonstrate that many-valued (MV) logic, specifically Lukasiewicz propositional logic, constitutes a suitable language for characterizing general CA as logical machines. This is done by interpolating CA transition functions to continuous piecewise linear functions, which, by virtue of the McNaughton theorem, yield formulae in MV logic characterizing the CA. Recognizing that deep rectified linear unit (ReLU) networks realize continuous piecewise linear functions, it follows that these formulae are naturally extracted from CA evolution traces by deep ReLU networks. A corresponding algorithm together with a software implementation is provided. Finally, we show that the dynamical behavior of CA can be realized by recurrent neural networks.

Cellular automata, many-valued logic, and deep neural networks

TL;DR

The paper addresses the problem of uncovering the logical rules governing cellular automata and extracting them from neural network traces. It develops a theoretical bridge between CA and Łukasiewicz MV logic, leveraging the McNaughton theorem to show that CA transition functions correspond to MV term functions, which deep ReLU networks can realize. By extending to divisible MV algebras (DMV), it accommodates rational coefficients in piecewise-linear interpolations and proves a universal NN representation for CA transitions, further embedding these into RNNs to realize CA dynamics. It also provides an algorithm to read out DMV formulas from trained networks, enabling identification of CA logic from evolution data, with a software implementation and explicit examples including retrieval of MV terms like τ = x_{-1} ⊕ x_0 ⊕ x_1. This work advances interpretable rule extraction from data-generated CA dynamics and links symbolic logic with neural architectures for dynamical systems.

Abstract

We develop a theory characterizing the fundamental capability of deep neural networks to learn, from evolution traces, the logical rules governing the behavior of cellular automata (CA). This is accomplished by first establishing a novel connection between CA and Lukasiewicz propositional logic. While binary CA have been known for decades to essentially perform operations in Boolean logic, no such relationship exists for general CA. We demonstrate that many-valued (MV) logic, specifically Lukasiewicz propositional logic, constitutes a suitable language for characterizing general CA as logical machines. This is done by interpolating CA transition functions to continuous piecewise linear functions, which, by virtue of the McNaughton theorem, yield formulae in MV logic characterizing the CA. Recognizing that deep rectified linear unit (ReLU) networks realize continuous piecewise linear functions, it follows that these formulae are naturally extracted from CA evolution traces by deep ReLU networks. A corresponding algorithm together with a software implementation is provided. Finally, we show that the dynamical behavior of CA can be realized by recurrent neural networks.
Paper Structure (18 sections, 17 theorems, 145 equations, 9 figures, 4 tables)

This paper contains 18 sections, 17 theorems, 145 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Cellular automata
  3. One-dimensional binary CA
  4. Previous work
  5. Outline of the paper
  6. Many-valued logic and cellular automata
  7. Many-valued algebras
  8. MV algebras and CA
  9. Recurrent neural networks realize CA dynamics
  10. DMV algebras and ReLU neural networks
  11. Realizing the dynamical behavior of CA
  12. Identification of CA logic from trained neural networks
  13. Interpolation
  14. Uniqueness properties of DMV formula extraction
  15. Extracting DMV formulae from trained networks
  16. ...and 3 more sections

Key Result

Lemma 2.1

Consider a CA with cellular space dimension $d \in \mathbb{N}$, neighborhood size $n \in \mathbb{N}$, state set $K = \{0,1\}$, and transition function $f:K^n \rightarrow K$ specified in the form of a (Boolean) truth table. There exists an MV term $\tau(x_1,\ldots \space, x_n)$ with associated term u

Figures (9)

  • Figure 1: Cell $z$ and its neighborhood.
  • Figure 2: Steps of the evolution of the Game of Life. Black cells are live; white cells are dead.
  • Figure 3: The state $1$ is indicated by black cells, the state $0$ by white cells.
  • Figure 4: The transition function $f$ and the associated function $f_{\text{c}}$ in Example \ref{['example McNaughton is not enough']}.
  • Figure 5: The transition function $g$ and the continuous function $g_{\text{c}}$ in Example \ref{['example McNaughton is not enough']}.
  • ...and 4 more figures

Theorems & Definitions (50)

  • Definition 1.1: Cellular automaton
  • Remark
  • Example 1.1
  • Example 1.2: Game of Life
  • Definition 1.2
  • Definition 2.1
  • Definition 2.2: MV term
  • Definition 2.3: Term function
  • Definition 2.4: Boolean algebra
  • Lemma 2.1
  • ...and 40 more