Category-Theoretical and Topos-Theoretical Frameworks in Machine Learning: A Survey
Yiyang Jia, Guohong Peng, Zheng Yang, Tianhao Chen
TL;DR
The survey addresses how category theory and topos theory can systematize machine learning, contrasting low-order (functorial component) frameworks for gradient-based, probabilistic, and invariant learning with high-order topos-based approaches that capture local–global data relations and causal reasoning. It outlines a modular gradient-based learning framework built from Para, Lens, and CRDC, and demonstrates how these components compose to model optimizers, losses, and network updates, including practical implementations and experiments. Additionally, it discusses probability-based learning via Markov categories and Bayesian inverses, and surveys topos-theoretic semantics, stacks, and classifying topoi as a foundation for interpretable and semantically grounded learning. The work highlights two avenues for advancement: developing coherent, compositional foundations that unify architectures across gradients, inference, and semantic layers, and further integrating high-level topos-theoretic semantics to illuminate local-global data interactions and causal structure in learning systems.
Abstract
In this survey, we provide an overview of category theory-derived machine learning from four mainstream perspectives: gradient-based learning, probability-based learning, invariance and equivalence-based learning, and topos-based learning. For the first three topics, we primarily review research in the past five years, updating and expanding on the previous survey by Shiebler et al.. The fourth topic, which delves into higher category theory, particularly topos theory, is surveyed for the first time in this paper. In certain machine learning methods, the compositionality of functors plays a vital role, prompting the development of specific categorical frameworks. However, when considering how the global properties of a network reflect in local structures and how geometric properties are expressed with logic, the topos structure becomes particularly significant and profound.