Autonomous Learning with High-Dimensional Computing Architecture Similar to von Neumann's

TL;DR

This work addresses how cognition and autonomous learning might be realized with high-dimensional vector processing that mirrors brain-inspired memory architectures. It proposes a von Neumann–like computer operating on $H$-dimensional vectors with a dedicated instruction set, a cerebellum-like long-term associative memory, and a working-memory focus that binds perception to action. Key contributions include the formalization of vector binding via addition, coordinatewise multiplication (Hadamard product), and permutation, plus a similarity measure from the dot product, enabling the encoding and retrieval of structured data and sequences in superposition; a clear separation of short-term focus and long-term storage; and a pathway to energy-efficient, in-memory learning for robotics. The approach aims to unify symbolic and probabilistic processing within a biologically plausible, scalable high-dimensional framework, with potential impacts on autonomous robotics, language, and cognitive modeling. By leveraging massive parallelism and robust superpositional representations, the theory promises practical, scalable experiments and hardware realizations that could approach brain-like efficiency. $H$-dimensional vector representations and the focus construct underpin a unified model of perception, memory, and action that blends psychology, neuroscience, and traditional computing concepts.

Abstract

We model human and animal learning by computing with high-dimensional vectors (H = 10,000 for example). The architecture resembles traditional (von Neumann) computing with numbers, but the instructions refer to vectors and operate on them in superposition. The architecture includes a high-capacity memory for vectors, analogue of the random-access memory (RAM) for numbers. The model's ability to learn from data reminds us of deep learning, but with an architecture closer to biology. The architecture agrees with an idea from psychology that human memory and learning involve a short-term working memory and a long-term data store. Neuroscience provides us with a model of the long-term memory, namely, the cortex of the cerebellum. With roots in psychology, biology, and traditional computing, a theory of computing with vectors can help us understand how brains compute. Application to learning by robots seems inevitable, but there is likely to be more, including language. Ultimately we want to compute with no more material and energy than used by brains. To that end, we need a mathematical theory that agrees with psychology and biology, and is suitable for nanotechnology. We also need to exercise the theory in large-scale experiments. Computing with vectors is described here in terms familiar to us from traditional computing with numbers.