A Path to Universal Neural Cellular Automata

TL;DR

The paper investigates whether neural cellular automata (NCA) trained by gradient descent can realize universal computation in a continuous CA substrate. It introduces a two-tier architecture separating a mutable computational state and an immutable hardware substrate, and defines an end-to-end training regime across diverse matrix-operator tasks. It demonstrates that NCAs can learn fundamental primitives such as matrix translation, multiplication, and transposition, and can emulate a neural network to solve MNIST within the CA state, indicating a path to analog general-purpose computation. The work's graph-based hardware hypernetwork and task-composition ideas point to scalable, hardware-aware automation of universal computation in continuous systems.

Abstract

Cellular automata have long been celebrated for their ability to generate complex behaviors from simple, local rules, with well-known discrete models like Conway's Game of Life proven capable of universal computation. Recent advancements have extended cellular automata into continuous domains, raising the question of whether these systems retain the capacity for universal computation. In parallel, neural cellular automata have emerged as a powerful paradigm where rules are learned via gradient descent rather than manually designed. This work explores the potential of neural cellular automata to develop a continuous Universal Cellular Automaton through training by gradient descent. We introduce a cellular automaton model, objective functions and training strategies to guide neural cellular automata toward universal computation in a continuous setting. Our experiments demonstrate the successful training of fundamental computational primitives - such as matrix multiplication and transposition - culminating in the emulation of a neural network solving the MNIST digit classification task directly within the cellular automata state. These results represent a foundational step toward realizing analog general-purpose computers, with implications for understanding universal computation in continuous dynamics and advancing the automated discovery of complex cellular automata behaviors via machine learning.

Figures (7)

• Figure 1: Schematic of our architecture, showing the distinction between mutable (computational) and immutable (hardware) states.
• Figure 2: Monolithic hardware configurations for 3 different sub-tasks. We plot a PCA projection of the hidden channels to be able to display them as RGB. Colors thus does not have direct functional relevance. Overlayed are schematics of the (fixed) input-output transformations that each hardware was optimized on.
• Figure 3: Modular hardware configurations for 3 different sub-tasks, and (now varying) different matrix sizes and placement.
• Figure 4: NCA emulates a neural network. Inputs show a batch of flattened MNIST images, alongside the weight matrix of a pre-trained single-layer linear classifier. We decompose this matrix multiplication into sub-blocks, that can be directly emulated in parallel by the NCA. Results are fetched from NCA states and aggregated back into logits (figure shows the first 32 outputs only for readability). We compute accuracy by taking the logits argmax per batch and comparing with labels.
• Figure 5: Examples of an Out of Distribution Task: the NCA needs to distribute a matrix in all corners in a larger grid than the one seen during training.