Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equivariant neural networks and equivarification

Erkao Bao, Jingcheng Lu, Linqi Song, Nathan Hart-Hodgson, William Parson, Yanheng Zhou

TL;DR

The paper tackles the challenge of hard-coding symmetries into neural networks by introducing equivarification, a universal framework that lifts any map to a $G$-equivariant form using a finite group action on inputs. It formalizes the lift to $X\to ext{Map}(G,Z)$ and develops both global and layer-wise constructions, showing that $G$-CNNs emerge as a special case and that parameter sharing preserves model size. The authors demonstrate the practicality of the approach through extensive experiments on MNIST, SVHN, and CIFAR-10, revealing competitive or superior performance to data augmentation, especially under known symmetry transformations and in data-scarce regimes. The work highlights that approximate equivariance (via larger groups with interpolation) can still yield meaningful robustness, offering a scalable and architecture-agnostic path to symmetry-aware deep learning.

Abstract

Equivariant neural networks are a class of neural networks designed to preserve symmetries inherent in the data. In this paper, we introduce a general method for modifying a neural network to enforce equivariance, a process we refer to as equivarification. We further show that group convolutional neural networks (G-CNNs) arise as a special case of our framework.

Equivariant neural networks and equivarification

TL;DR

The paper tackles the challenge of hard-coding symmetries into neural networks by introducing equivarification, a universal framework that lifts any map to a -equivariant form using a finite group action on inputs. It formalizes the lift to and develops both global and layer-wise constructions, showing that -CNNs emerge as a special case and that parameter sharing preserves model size. The authors demonstrate the practicality of the approach through extensive experiments on MNIST, SVHN, and CIFAR-10, revealing competitive or superior performance to data augmentation, especially under known symmetry transformations and in data-scarce regimes. The work highlights that approximate equivariance (via larger groups with interpolation) can still yield meaningful robustness, offering a scalable and architecture-agnostic path to symmetry-aware deep learning.

Abstract

Equivariant neural networks are a class of neural networks designed to preserve symmetries inherent in the data. In this paper, we introduce a general method for modifying a neural network to enforce equivariance, a process we refer to as equivarification. We further show that group convolutional neural networks (G-CNNs) arise as a special case of our framework.

Paper Structure

This paper contains 15 sections, 3 theorems, 60 equations, 10 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Equivarification
  4. Application of equivarification to neural networks
  5. A multi-view construction
  6. Layer-by-layer equivarification
  7. G-CNNs as a special case of equivarification
  8. Numerical experiments
  9. Label and loss function
  10. MNIST
  11. SVHN
  12. CIFAR10
  13. Group refinement and equivariance error
  14. Conclusion
  15. More on Equivarification

Key Result

Lemma 3.2

Given any map $F: X \to Z$, there exists a unique $G$-equivariant map $\widehat{F}: X \to \operatorname{Map}(G, Z)$ such that

Figures (10)

  • Figure 1: Every map $F: X \to Z$ admits a unique $G$-equivariant lift $\widehat{F}$ such that $p \circ \widehat{F} = F$.
  • Figure 2: Structure of layer-by-layer equivarification. Each layer performs both a standard mapping $F_i$ and its lifted equivariant version $\widehat{F}_i$, followed by projection $p_i: X_i \to \widehat{X}_i$.
  • Figure 3: An equivariant CNN structure with three convolution layers on MNIST. Each layer is lifted via multi-view evaluation over group-transformed inputs. The final classification is obtained via argmax over $\widehat{X}_5 = \mathbb{R}^{40}$, which encodes 10 classes $\times$ 4 group elements.
  • Figure 4: For any $F:X\to Y$, we construct a map $\widehat{X}\to\widehat{Y}$.
  • Figure 5: digit $7$, angle $0^\circ$
  • ...and 5 more figures

Theorems & Definitions (18)

  • Definition 2.1: Group Action
  • Definition 2.2: Equivariance and Invariance
  • Example 2.3: Equivariance in Image Classification
  • Definition 3.1
  • Lemma 3.2
  • proof
  • Remark 3.3
  • Definition 3.4: $G$-Equivarification
  • Example 3.5
  • Definition 4.1
  • ...and 8 more