Decomposition of Equivariant Maps via Invariant Maps: Application to Universal Approximation under Symmetry

TL;DR

This work establishes a fundamental link between invariant and equivariant maps under a group $G$ by proving a bijection between $G$-equivariant maps $V^X\to W^Y$ and tuples of stabilizer-invariant maps on the orbits of $Y$, via the stabilizer subgroups $H_i = \mathrm{Stab}_G(y_i)$. This decomposition allows universal $G$-equivariant architectures to be built from already universal invariant networks, through a constructive map $\Psi$ that aggregates invariant components, and it provides inequalities relating the numbers of parameters needed for invariant versus equivariant approximation. The paper also analyzes approximation rates for $G$-equivariant ReLU networks in Hölder spaces, generalizing results from the symmetric group to arbitrary finite groups, and discusses the structural implications of a new hidden-layer action $\ast$ induced by the decomposition. Overall, the framework offers a principled, scalable approach to symmetry-aware neural architectures with potential parameter-efficiency advantages and broad applicability across discrete and continuous groups.

Abstract

In this paper, we develop a theory about the relationship between invariant and equivariant maps with regard to a group $G$. We then leverage this theory in the context of deep neural networks with group symmetries in order to obtain novel insight into their mechanisms. More precisely, we establish a one-to-one relationship between equivariant maps and certain invariant maps. This allows us to reduce arguments for equivariant maps to those for invariant maps and vice versa. As an application, we propose a construction of universal equivariant architectures built from universal invariant networks. We, in turn, explain how the universal architectures arising from our construction differ from standard equivariant architectures known to be universal. Furthermore, we explore the complexity, in terms of the number of free parameters, of our models, and discuss the relation between invariant and equivariant networks' complexity. Finally, we also give an approximation rate for G-equivariant deep neural networks with ReLU activation functions for finite group G.

Figures (3)

• Figure 1: Commutative diagrams for comparing between a usual equivariant multi-layer architecture, and ours arising from Theorem \ref{['thm:approx']}. For each diagram, the two lines both represent the same multi-layer equivariant architecture with the $\varphi_i$ being the equivariant layers between hidden spaces. Each square encapsulating a $\circlearrowright$ symbol is a commutative diagram that represents the equivariance of the layer $\varphi_i$, i.e, the fact that $\varphi_i$ commutes with some action of the group. This means that, within these squares, from the top left to the bottom right the two possible paths are equal.
• Figure 2: The case of $G = S_n$ and $X = Y = \{1,2, \dots, n \}$. The $S_n$-orbit of $1\in Y$ is the whole set $Y$, thus $S_n$-equivariant map $F$ is determined by only one $\mathrm{Stab}_{S_n}(1)$-invariant map $\widetilde{F}_1$.
• Figure 3: Rotation equivariant map $F$ is determined the maps $\{ \widetilde{F}_i \}_{i \in I}$ where $I$ is the nonnegative real line $\mathbb{R}_{\geq 0}$. We can recover $F$ from $\{ \widetilde{F}_i \}_{i \in I}$ by "rotating" $\{\widetilde{F}_i(g\cdot )\}_{i\in I, g\in \mathrm{SO}_2(\mathbb{R})}$

Theorems & Definitions (30)

• Example 1
• Example 2
• Definition 1
• Example 3
• Proposition 1
• proof
• Theorem 1
• proof
• Theorem 2
• proof