CayleyNets: Graph Convolutional Neural Networks with Complex Rational Spectral Filters

TL;DR

CayleyNets address the challenge of scalable, band-focused graph convolution by introducing complex rational Cayley filters $g_{\mathbf{c},h}(\boldsymbol{\Delta})$ based on the Cayley transform $\mathcal{C}(h\boldsymbol{\Delta})$, which yields unitary operators and enables stable, localized spectral filtering. The framework supports a spectral zoom parameter $h$ to concentrate on narrow frequency bands, and uses Jacobi-based inversions to maintain linear complexity on sparse graphs. The authors provide analytic properties, localization guarantees, and complexity analyses, and validate the approach on MNIST, CORA, and MovieLens, showing improvements over ChebNet and other spectral CNNs, especially for low-order filters and band-focused tasks. This work advances scalable, transferable graph neural networks with strong control over spectral localization, benefiting large-scale graph learning applications in vision, text, and recommender systems.

Abstract

The rise of graph-structured data such as social networks, regulatory networks, citation graphs, and functional brain networks, in combination with resounding success of deep learning in various applications, has brought the interest in generalizing deep learning models to non-Euclidean domains. In this paper, we introduce a new spectral domain convolutional architecture for deep learning on graphs. The core ingredient of our model is a new class of parametric rational complex functions (Cayley polynomials) allowing to efficiently compute spectral filters on graphs that specialize on frequency bands of interest. Our model generates rich spectral filters that are localized in space, scales linearly with the size of the input data for sparsely-connected graphs, and can handle different constructions of Laplacian operators. Extensive experimental results show the superior performance of our approach, in comparison to other spectral domain convolutional architectures, on spectral image classification, community detection, vertex classification and matrix completion tasks.

Figures (6)

• Figure 1: Eigenvalues of the unnormalized Laplacian $h\boldsymbol{\Delta}_\mathrm{u}$ of the 15-communities graph mapped on the complex unit half-circle by means of Cayley transform with spectral zoom values (left-to-right) $h=0.1$, $1$, and $10$. The first 15 frequencies carrying most of the information about the communities are marked in red. Larger values of $h$ zoom (right) on the low frequency band.
• Figure 2: Filters (spatial domain, top and spectral domain, bottom) learned by CayleyNet (left) and ChebNet (center, right) on the MNIST dataset. Cayley filters are able to realize larger supports for the same order $r$.
• Figure 3: Left: synthetic 15-communities graph. Right: community detection accuracy of ChebNet and CayleyNet (top); normalized responses of four different filters learned by ChebNet (middle) and CayleyNet (bottom). Grey vertical lines represent the frequencies of the normalized Laplacian ($\tilde{\lambda} = 2\lambda_n^{-1} \lambda - 1$ for ChebNet and $C(\lambda) = (h\lambda - i)/(h\lambda + i)$ unrolled to a real line for CayleyNet). Note how thanks to spectral zoom property Cayley filters can focus on the band of low frequencies (dark grey lines) containing most of the information about communities.
• Figure 4: Community detection test accuracy as function of filter order $r$. Shown are exact matrix inversion (dashed) and approximate Jacobi with different number of iterations (colored). For reference, ChebNet is shown (dotted).
• Figure 5: ChebNet (blue) and CayleyNet (orange) test accuracies obtained on the CORA dataset for different polynomial orders. Polynomials with complex coefficients (top two) and real coefficients (bottom two) have been exploited with CayleyNet in the two analysis. Orders 1 to 6 have been used in both comparisons.

Theorems & Definitions (4)

• Proposition 1
• Definition 2: Exponential decay on graphs
• Remark 3
• Theorem 4