LifeGPT: Topology-Agnostic Generative Pretrained Transformer Model for Cellular Automata

TL;DR

LifeGPT, a decoder-only generative pretrained transformer model with rotary positional embedding and forgetful causal masking, capable of computing a single-timestep global state transition in Life on a toroidal grid without prior knowledge of grid size or boundary conditions is introduced.

Abstract

Conway's Game of Life (Life), a well known algorithm within the broader class of cellular automata (CA), exhibits complex emergent dynamics, with extreme sensitivity to initial conditions. Modeling and predicting such intricate behavior without explicit knowledge of the system's underlying topology presents a significant challenge, motivating the development of algorithms that can generalize across various grid configurations and boundary conditions. We develop a decoder-only generative pretrained transformer (GPT) model to solve this problem, showing that our model can simulate Life on a toroidal grid with no prior knowledge on the size of the grid, or its periodic boundary conditions (LifeGPT). LifeGPT is topology-agnostic with respect to its training data and our results show that a GPT model is capable of capturing the deterministic rules of a Turing-complete system with near-perfect accuracy, given sufficiently diverse training data. We also introduce the idea of an `autoregressive autoregressor' to recursively implement Life using LifeGPT. Our results pave the path towards true universal computation within a large language model framework, synthesizing of mathematical analysis with natural language processing, and probing AI systems for situational awareness about the evolution of such algorithms without ever having to compute them. Similar GPTs could potentially solve inverse problems in multicellular self-assembly by extracting CA-compatible rulesets from real-world biological systems to create new predictive models, which would have significant consequences for the fields of bioinspired materials, tissue engineering, and architected materials design.

Figures (17)

• Figure 1: A state transition diagram for a single cell in Life. $N$ represents the number of neighboring cells in state 1 (live cells). The two large circles represent the possible states of a cell. Arrows represent deterministic state transitions.
• Figure 1: Top row: the ground truth (GT) progression of Life, for an IC corresponding to $\langle\eta\rangle_{\mathrm{IC}}=0$. Middle row: LifeGPT's best prediction (Pred) using the ARAR method at temperature=0. Bottom row: the discrepancy between the ground truth and LifeGPT's predictions (Error), where blue indicates no discrepancy, and yellow indicates an incorrect cell predicted by LifeGPT.
• Figure 2: Multiple model performance benchmarks vs. epoch. (A) Training and validation losses. (B) Accuracy across varying temperatures.
• Figure 2: Top row: the ground truth (GT) progression of Life, for an IC corresponding to $\langle\eta\rangle_{\mathrm{IC}}=0.25$. Middle row: LifeGPT's best prediction (Pred) using the ARAR method at temperature=0. Bottom row: the discrepancy between the ground truth and LifeGPT's predictions (Error), where blue indicates no discrepancy, and yellow indicates an incorrect cell predicted by LifeGPT.
• Figure 3: ICs in the test set. Top row: ICs with increasing $\eta$ from left to right. Bottom row: ICs corresponding to known periodic and complex patterns in Life.