Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Group Downsampling with Equivariant Anti-aliasing

Md Ashiqur Rahman, Raymond A. Yeh

TL;DR

The paper tackles downsampling in group-equivariant networks by introducing a mathematically grounded framework for uniform subgroup downsampling on finite groups. It develops a Subgroup Sampling Theorem and an equivariant anti-aliasing operator, plus an algorithm to select appropriate subgroups at a given rate $R$. Empirical results on MNIST, CIFAR-10, and STL-10 show that subgroup subsampling with anti-aliasing preserves or improves accuracy and equivariance while reducing parameter counts. These contributions provide a principled path to building scalable, symmetry-preserving networks on general finite groups.

Abstract

Downsampling layers are crucial building blocks in CNN architectures, which help to increase the receptive field for learning high-level features and reduce the amount of memory/computation in the model. In this work, we study the generalization of the uniform downsampling layer for group equivariant architectures, e.g., G-CNNs. That is, we aim to downsample signals (feature maps) on general finite groups with anti-aliasing. This involves the following: (a) Given a finite group and a downsampling rate, we present an algorithm to form a suitable choice of subgroup. (b) Given a group and a subgroup, we study the notion of bandlimited-ness and propose how to perform anti-aliasing. Notably, our method generalizes the notion of downsampling based on classical sampling theory. When the signal is on a cyclic group, i.e., periodic, our method recovers the standard downsampling of an ideal low-pass filter followed by a subsampling operation. Finally, we conducted experiments on image classification tasks demonstrating that the proposed downsampling operation improves accuracy, better preserves equivariance, and reduces model size when incorporated into G-equivariant networks

Group Downsampling with Equivariant Anti-aliasing

TL;DR

The paper tackles downsampling in group-equivariant networks by introducing a mathematically grounded framework for uniform subgroup downsampling on finite groups. It develops a Subgroup Sampling Theorem and an equivariant anti-aliasing operator, plus an algorithm to select appropriate subgroups at a given rate . Empirical results on MNIST, CIFAR-10, and STL-10 show that subgroup subsampling with anti-aliasing preserves or improves accuracy and equivariance while reducing parameter counts. These contributions provide a principled path to building scalable, symmetry-preserving networks on general finite groups.

Abstract

Downsampling layers are crucial building blocks in CNN architectures, which help to increase the receptive field for learning high-level features and reduce the amount of memory/computation in the model. In this work, we study the generalization of the uniform downsampling layer for group equivariant architectures, e.g., G-CNNs. That is, we aim to downsample signals (feature maps) on general finite groups with anti-aliasing. This involves the following: (a) Given a finite group and a downsampling rate, we present an algorithm to form a suitable choice of subgroup. (b) Given a group and a subgroup, we study the notion of bandlimited-ness and propose how to perform anti-aliasing. Notably, our method generalizes the notion of downsampling based on classical sampling theory. When the signal is on a cyclic group, i.e., periodic, our method recovers the standard downsampling of an ideal low-pass filter followed by a subsampling operation. Finally, we conducted experiments on image classification tasks demonstrating that the proposed downsampling operation improves accuracy, better preserves equivariance, and reduces model size when incorporated into G-equivariant networks

Paper Structure

This paper contains 29 sections, 4 theorems, 42 equations, 16 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Method
  5. Uniform group subsampling
  6. The Subgroup Sampling theorem for signals on groups
  7. Experiments and Evaluations
  8. Emperical validation for Claim \ref{['claim:subgroup_sampling_theorem']}
  9. Image Classification
  10. Conclusion
  11. Group Theory Preliminaries
  12. Group
  13. Group Representation
  14. Invariant and Equivariant Maps
  15. Illustration challenges in Subgroup Subsampling
  16. ...and 14 more sections

Key Result

Lemma 1

For the set $G^\downarrow$ returned by Alg. alg:sampling, $v \in G^\downarrow$ if and only if $v$ can be expressed as a product of the elements of the set $S^\downarrow = (S/\{s_d\}) \cup \{s_d^R\}$.

Figures (16)

  • Figure 1: (a) Cayley graph of the group with generator $\Delta t$. (b) Edges corresponding to the generator $\Delta t =1$ are removed (dotted edges) and new edges corresponding to the element $2 \Delta t$ are added. (c) The resultant cyclic subgroup of size $2$ obtain by the traversing the new graph from node $0$.
  • Figure 2: Subsampling group $D_8$ along the generator $s$ (on left) and $r$ (on right). The edges corresponding to the subsampling generators are dotted in the Cayley graph.
  • Figure 3: Visualization of the smoothing filter (${\mathcal{P}}_{\mathcal{M}} \delta_G$) used in the anti-aliasing operation for subgroup subsampling. The vertical bar corresponds to the value of the filter at each node, with the downward bars indicating negative values.
  • Figure : Uniform group subsampling
  • Figure : Empirical Validation of Claim \ref{['claim:subgroup_sampling_theorem']}. We report the recon. error with / (and without) the anti-aliasing operation. Anti-aliasing achieves zero recon. error up to numerical precision.
  • ...and 11 more figures

Theorems & Definitions (17)

  • Example 1
  • Lemma 1
  • Lemma 2
  • Claim 1
  • proof
  • Example 2
  • Claim 2
  • proof
  • Example 3
  • Lemma 2
  • ...and 7 more