Manifold limit for the training of shallow graph convolutional neural networks
Johanna Tengler, Christoph Brune, José A. Iglesias
TL;DR
This work addresses the problem of training shallow graph convolutional neural networks (GCNNs) on proximity graphs constructed from samples lying on an underlying smooth manifold. It recasts GCNN training in a continuum setting by representing parameters as measures and proving that regularized empirical risk minimizers Γ-converge to a continuum variational problem, with minimizers converging in the weak-* topology and neural responses converging uniformly on compact input sets. The approach leverages spectral definitions of graph convolutions, low-frequency graph-Laplacian spectra approximating the Laplace-Beltrami operator, and a carefully designed spectral cutoff $K(n)$ to ensure mesh and sample independence in the limit. The results formalize the consistency of training across graph resolutions, linking discrete training problems to a well-posed continuum limit and providing a rigorous foundation for transferability in geometric learning systems.
Abstract
We study the discrete-to-continuum consistency of the training of shallow graph convolutional neural networks (GCNNs) on proximity graphs of sampled point clouds under a manifold assumption. Graph convolution is defined spectrally via the graph Laplacian, whose low-frequency spectrum approximates that of the Laplace-Beltrami operator of the underlying smooth manifold, and shallow GCNNs of possibly infinite width are linear functionals on the space of measures on the parameter space. From this functional-analytic perspective, graph signals are seen as spatial discretizations of functions on the manifold, which leads to a natural notion of training data consistent across graph resolutions. To enable convergence results, the continuum parameter space is chosen as a weakly compact product of unit balls, with Sobolev regularity imposed on the output weight and bias, but not on the convolutional parameter. The corresponding discrete parameter spaces inherit the corresponding spectral decay, and are additionally restricted by a frequency cutoff adapted to the informative spectral window of the graph Laplacians. Under these assumptions, we prove $Γ$-convergence of regularized empirical risk minimization functionals and corresponding convergence of their global minimizers, in the sense of weak convergence of the parameter measures and uniform convergence of the functions over compact sets. This provides a formalization of mesh and sample independence for the training of such networks.