Memory makes computation universal, remember?
Erik Garrison
TL;DR
Problem: What drives universal computation across biology and AI? The paper argues memory—specifically recursive state maintenance and reliable history access—enables universal computation across scales. Approach: formalizes these capabilities, provides a constructive simulation of a Universal Turing Machine $M$ by a system $S$ with at most $O(\log t)$ overhead, encoding configurations as $s=(q,p,a,t)$ and history as $h_p=(p,a_p,t_p)$. Findings: memory-enabled computation accounts for sequential capability in parallel systems (neural networks, cells, language models) and yields practical guidance for AI architecture, e.g., chain-of-thought, ARC, and o3 results. Significance: shifts focus from purely increasing processing power to reinforcing memory mechanisms and state maintenance to achieve reliable, scalable computation across domains.
Abstract
Recent breakthroughs in AI capability have been attributed to increasingly sophisticated architectures and alignment techniques, but a simpler principle may explain these advances: memory makes computation universal. Memory enables universal computation through two fundamental capabilities: recursive state maintenance and reliable history access. We formally prove these requirements are both necessary and sufficient for universal computation. This principle manifests across scales, from cellular computation to neural networks to language models. Complex behavior emerges not from sophisticated processing units but from maintaining and accessing state across time. We demonstrate how parallel systems like neural networks achieve universal computation despite limitations in their basic units by maintaining state across iterations. This theoretical framework reveals a universal pattern: computational advances consistently emerge from enhanced abilities to maintain and access state rather than from more complex basic operations. Our analysis unifies understanding of computation across biological systems, artificial intelligence, and human cognition, reminding us that humanity's own computational capabilities have evolved in step with our technical ability to remember through oral traditions, writing, and now computing.