Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equivariant Machine Learning on Graphs with Nonlinear Spectral Filters

Ya-Wei Eileen Lin, Ronen Talmon, Ron Levie

TL;DR

This paper proposes nonlinear spectral filters that are fully equivariant to graph functional shifts and show that they have universal approximation properties and demonstrates the superior performance of NLSFs over existing spectral GNNs in node and graph classification benchmarks.

Abstract

Equivariant machine learning is an approach for designing deep learning models that respect the symmetries of the problem, with the aim of reducing model complexity and improving generalization. In this paper, we focus on an extension of shift equivariance, which is the basis of convolution networks on images, to general graphs. Unlike images, graphs do not have a natural notion of domain translation. Therefore, we consider the graph functional shifts as the symmetry group: the unitary operators that commute with the graph shift operator. Notably, such symmetries operate in the signal space rather than directly in the spatial space. We remark that each linear filter layer of a standard spectral graph neural network (GNN) commutes with graph functional shifts, but the activation function breaks this symmetry. Instead, we propose nonlinear spectral filters (NLSFs) that are fully equivariant to graph functional shifts and show that they have universal approximation properties. The proposed NLSFs are based on a new form of spectral domain that is transferable between graphs. We demonstrate the superior performance of NLSFs over existing spectral GNNs in node and graph classification benchmarks.

Equivariant Machine Learning on Graphs with Nonlinear Spectral Filters

TL;DR

This paper proposes nonlinear spectral filters that are fully equivariant to graph functional shifts and show that they have universal approximation properties and demonstrates the superior performance of NLSFs over existing spectral GNNs in node and graph classification benchmarks.

Abstract

Equivariant machine learning is an approach for designing deep learning models that respect the symmetries of the problem, with the aim of reducing model complexity and improving generalization. In this paper, we focus on an extension of shift equivariance, which is the basis of convolution networks on images, to general graphs. Unlike images, graphs do not have a natural notion of domain translation. Therefore, we consider the graph functional shifts as the symmetry group: the unitary operators that commute with the graph shift operator. Notably, such symmetries operate in the signal space rather than directly in the spatial space. We remark that each linear filter layer of a standard spectral graph neural network (GNN) commutes with graph functional shifts, but the activation function breaks this symmetry. Instead, we propose nonlinear spectral filters (NLSFs) that are fully equivariant to graph functional shifts and show that they have universal approximation properties. The proposed NLSFs are based on a new form of spectral domain that is transferable between graphs. We demonstrate the superior performance of NLSFs over existing spectral GNNs in node and graph classification benchmarks.
Paper Structure (60 sections, 8 theorems, 82 equations, 5 figures, 17 tables)

This paper contains 60 sections, 8 theorems, 82 equations, 5 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Linear Graph Signal Processing
  4. Equivariant GNNs
  5. Nonlinear Spectral Graph Filters
  6. Translation Equivariance of CNNs and GNNs
  7. Graph Functional Symmetries and Their Relaxations
  8. Analysis and Synthesis
  9. Definitions of Nonlinear Spectral Filters
  10. Laplacian Attention NLSFs
  11. Theoretical Properties of Nonlinear Spectral Filters
  12. Complexity of NLSFs
  13. Equivariance of Node-level NLSFs
  14. Universal Approximation and Expressivity
  15. Uniform Approximation of GSOs by Their Leading Eigenvectors
  16. ...and 45 more sections

Key Result

Proposition 4.1

Index NLSFs in Eq. eq:spec_frequency_learning_ind are equivariant to the graph functional shifts $\mathcal{U}_{{\boldsymbol{\Delta}}}$, and Value NLSFs in Eq. eq:spec_frequency_learning_val are equivariant to the relaxed graph functional shifts $\mathcal{U}^g_{{\boldsymbol{\Delta}}}$.

Figures (5)

  • Figure 1: Illustration of nonlinear spectral filters for equivariant machine learning on graphs. Given a graph $G$, the node features $\mathbf{X}$ are projected onto eigenspaces (analysis $\mathcal{A}$). The function $\Psi$ map a sequence of frequency coefficients to a sequence of frequency coefficients. The coefficients are synthesized to the graph domain using the using $\mathcal{S}$.
  • Figure 2: Illustration of Laplacian attention NLSFs. An attention mechanism is applied to both Index NLSFs and Value NLSFs, enabling the adaptive selection of the most appropriate parameterization.
  • Figure 3: Comparison of classical and functional translation of a Gaussian signal. Top Row (Classical Translation): The Gaussian signal moves uniformly across the grid without changing shape. Bottom Row (Functional Translation): The Gaussian signal translates as low-frequency components move at different speeds than high-frequency components, demonstrating relaxed symmetry.
  • Figure 4: Approximate a standard translation by functional translation on a perturbed graph.
  • Figure 5: Illustration of dyadic sub-bands for $r= \frac{1}{2}$ and $S = 4$.

Theorems & Definitions (15)

  • Definition 3.1: Graph Functional Shifts
  • Definition 3.2: Relaxed Functional Shifts
  • Definition 3.3: Leading Functional Shifts
  • Proposition 4.1
  • Lemma 4.1
  • Theorem 4.1
  • Lemma D.1
  • proof
  • Lemma D.2
  • proof
  • ...and 5 more