Categorical Equivariant Deep Learning: Category-Equivariant Neural Networks and Universal Approximation Theorems
Yoshihiro Maruyama
TL;DR
<1> The paper develops category-equivariant neural networks (CENNs), recasting equivariance as naturality in a topological category with Radon measures to unify diverse symmetry structures beyond groups. <2> Data are modeled as functors $X,Y:\mathcal{C}^{op}\to \mathbf{Vect}$ and neural layers as natural transformations, with linear layers realized as category convolutions constrained by the integrated naturality and Carathéodory regularity. <3> It proves a universal approximation theorem: finite-depth $\mathsf{CENN}_\alpha(X,Y)$ are dense in the space of continuous equivariant maps $\mathrm{EqvCont}(X,Y)$ under the compact-open, finite-object topology, via a constructive, multi-step proof. <4> The framework subsumes group/groupoid, poset/lattice, graph, and cellular-sheaf architectures, offering a principled, compositional approach to symmetry- and structure-aware learning with broad potential impact across geometry, logic, and dynamics.
Abstract
We develop a theory of category-equivariant neural networks (CENNs) that unifies group/groupoid-equivariant networks, poset/lattice-equivariant networks, graph and sheaf neural networks. Equivariance is formulated as naturality in a topological category with Radon measures. Formulating linear and nonlinear layers in the categorical setup, we prove the equivariant universal approximation theorem in the general setting: the class of finite-depth CENNs is dense in the space of continuous equivariant transformations. We instantiate the framework for groups/groupoids, posets/lattices, graphs and cellular sheaves, deriving universal approximation theorems for them in a systematic manner. Categorical equivariant deep learning thus allows us to expand the horizons of equivariant deep learning beyond group actions, encompassing not only geometric symmetries but also contextual and compositional symmetries.