Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network

Risi Kondor, Zhen Lin, Shubhendu Trivedi

TL;DR

This work advances spherical convolutional neural networks by delivering a fully Fourier-space, $SO(3)$-equivariant architecture that uses the Clebsch--Gordan transform as the sole nonlinearity, eliminating the need for repeated real-space Fourier transforms. By representing activations as collections of irreducible fragments and enforcing covariance through per-irrep linear maps, the network achieves exact rotational invariance while remaining computationally efficient through channel management. The approach demonstrates strong performance across rotated MNIST on the sphere, QM7 atomization energy prediction, and SHREC3D, illustrating both robustness to rotations and applicability to diverse 3D tasks. The framework also generalizes to other compact groups, offering a general formalism for fully Fourier neural networks with equivariance properties beyond $SO(3)$.

Abstract

Recent work by Cohen \emph{et al.} has achieved state-of-the-art results for learning spherical images in a rotation invariant way by using ideas from group representation theory and noncommutative harmonic analysis. In this paper we propose a generalization of this work that generally exhibits improved performace, but from an implementation point of view is actually simpler. An unusual feature of the proposed architecture is that it uses the Clebsch--Gordan transform as its only source of nonlinearity, thus avoiding repeated forward and backward Fourier transforms. The underlying ideas of the paper generalize to constructing neural networks that are invariant to the action of other compact groups.

Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network

TL;DR

This work advances spherical convolutional neural networks by delivering a fully Fourier-space, -equivariant architecture that uses the Clebsch--Gordan transform as the sole nonlinearity, eliminating the need for repeated real-space Fourier transforms. By representing activations as collections of irreducible fragments and enforcing covariance through per-irrep linear maps, the network achieves exact rotational invariance while remaining computationally efficient through channel management. The approach demonstrates strong performance across rotated MNIST on the sphere, QM7 atomization energy prediction, and SHREC3D, illustrating both robustness to rotations and applicability to diverse 3D tasks. The framework also generalizes to other compact groups, offering a general formalism for fully Fourier neural networks with equivariance properties beyond .

Abstract

Recent work by Cohen \emph{et al.} has achieved state-of-the-art results for learning spherical images in a rotation invariant way by using ideas from group representation theory and noncommutative harmonic analysis. In this paper we propose a generalization of this work that generally exhibits improved performace, but from an implementation point of view is actually simpler. An unusual feature of the proposed architecture is that it uses the Clebsch--Gordan transform as its only source of nonlinearity, thus avoiding repeated forward and backward Fourier transforms. The underlying ideas of the paper generalize to constructing neural networks that are invariant to the action of other compact groups.

Paper Structure

This paper contains 14 sections, 3 theorems, 22 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Convolutions on the sphere
  3. Fourier space filters and activations
  4. Generalized spherical CNNs
  5. Covariant linear transformations
  6. Covariant nonlinearities: the Clebsch--Gordan transform
  7. Limiting the number of channels
  8. Final invariant layer
  9. Summary of algorithm
  10. Experiments
  11. Rotated MNIST on the Sphere
  12. Atomization Energy Prediction
  13. 3D Shape Recognition
  14. Conclusion

Key Result

Proposition 1

Let ${f\colon S^2\to\mathbb{C}}$ be an activation function that under the action of a rotation $R$ transforms as (eq: rotation), and let ${h\colon S^2\to\mathbb{C}}$ be a filter. Then, each Fourier component of the cross correlation (eq: S^2 xcorr) transforms as Similarly, if ${f',h'\colon\mathrm{SO}(3)\to\mathbb{C}}$, then ${\widehat{h'\space\star\space f'}}$ (as defined in (eq: xcorr Fourier2))

Figures (1)

  • Figure 1: Schematic of a single layer of the Clebsch--Gordan network.

Theorems & Definitions (4)

  • Proposition 1
  • Definition 1
  • Proposition 2
  • Lemma 3