From Dionysius Emerges Apollo -- Learning Patterns and Abstractions from Perceptual Sequences

TL;DR

This work investigates how structured representations emerge from perceptual sequences, proposing chunking and abstraction as normative, scalable principles that enable learning, generalization, and prediction. It advances this view through three interconnected strands: (i) empirical studies of chunking in serial reaction time tasks that reveal adaptive chunking and speed–accuracy trade-offs; (ii) a hierarchical chunking framework (HCM) that factorizes sequences into reusable primitives and supports compositionality across one- and multi-dimensional domains; and (iii) a move from concrete to abstract representations via motif learning and a hierarchical variable model (HVM) that integrates abstraction with chunking to uncover invariant patterns. The results show that humans exploit both concrete and abstract patterns for memory and transfer, and that the proposed normative models align with human behavior and, in some contexts, outperform large language models in capturing abstraction-driven generalization. Collectively, the thesis argues that chunking and abstraction underpin the acquisition of structured knowledge from hierarchical perceptual data, offering a principled bridge between cognitive science and AI with implications for memory, learning, and transfer across domains.

Abstract

Cognition swiftly breaks high-dimensional sensory streams into familiar parts and uncovers their relations. Why do structures emerge, and how do they enable learning, generalization, and prediction? What computational principles underlie this core aspect of perception and intelligence? A sensory stream, simplified, is a one-dimensional sequence. In learning such sequences, we naturally segment them into parts -- a process known as chunking. In the first project, I investigated factors influencing chunking in a serial reaction time task and showed that humans adapt to underlying chunks while balancing speed and accuracy. Building on this, I developed models that learn chunks and parse sequences chunk by chunk. Normatively, I proposed chunking as a rational strategy for discovering recurring patterns and nested hierarchies, enabling efficient sequence factorization. Learned chunks serve as reusable primitives for transfer, composition, and mental simulation -- letting the model compose the new from the known. I demonstrated this model's ability to learn hierarchies in single and multi-dimensional sequences and highlighted its utility for unsupervised pattern discovery. The second part moves from concrete to abstract sequences. I taxonomized abstract motifs and examined their role in sequence memory. Behavioral evidence suggests that humans exploit pattern redundancies for compression and transfer. I proposed a non-parametric hierarchical variable model that learns both chunks and abstract variables, uncovering invariant symbolic patterns. I showed its similarity to human learning and compared it to large language models. Taken together, this thesis suggests that chunking and abstraction as simple computational principles enable structured knowledge acquisition in hierarchically organized sequences, from simple to complex, concrete to abstract.