Theoretical guarantees for the advantage of GNNs over NNs in generalizing bandlimited functions on Euclidean cubes

TL;DR

The paper tackles the problem of understanding when GNNs can generalize bandlimited functions on a Euclidean cube better than standard NNs. It develops a sampling-theoretic construction that embeds a regularized, truncated Whittaker–Kotel’nikov–Shannon reconstruction into a GNN, yielding explicit weight and layer counts of $O_d((\,log\varepsilon^{-1})^d)$ for $ ext{ε}$-accuracy. The main contributions are three theorems: a uniform-sampling reconstruction (Theorem samp), a 1D proxy-GNN achieving sampling-based interpolation (Theorem proxyGNN), and a high-dimensional GNN generalizing bandlimited functions on $[-1,1]^d$ (Theorem actualGNN), with detailed proofs and a realizable GNN architecture. The results provide a principled, pre-trained network grounded in classical harmonic analysis, offering a rigorous link between GNNs and sampling theory, and illustrating a potentially sharper generalization performance in multi-dimensional settings. This work thus advances the theoretical understanding of GNN expressiveness and generalization for smooth, bandlimited targets in graph-based learning contexts.

Abstract

Graph Neural Networks (GNNs) have emerged as formidable resources for processing graph-based information across diverse applications. While the expressive power of GNNs has traditionally been examined in the context of graph-level tasks, their potential for node-level tasks, such as node classification, where the goal is to interpolate missing node labels from the observed ones, remains relatively unexplored. In this study, we investigate the proficiency of GNNs for such classifications, which can also be cast as a function interpolation problem. Explicitly, we focus on ascertaining the optimal configuration of weights and layers required for a GNN to successfully interpolate a band-limited function over Euclidean cubes. Our findings highlight a pronounced efficiency in utilizing GNNs to generalize a bandlimited function within an $\varepsilon$-error margin. Remarkably, achieving this task necessitates only $O_d((\log\varepsilon^{-1})^d)$ weights and $O_d((\log\varepsilon^{-1})^d)$ training samples. We explore how this criterion stacks up against the explicit constructions of currently available Neural Networks (NNs) designed for similar tasks. Significantly, our result is obtained by drawing an innovative connection between the GNN structures and classical sampling theorems. In essence, our pioneering work marks a meaningful contribution to the research domain, advancing our understanding of the practical GNN applications.

Figures (3)

• Figure 1: Toy example of graph used in GNN when learning a bandlimited function on $[-1, 1]^2$.
• Figure 2: The graph of $\pi_7(t)$ on $[0,7]$
• Figure 3: The graph of $t\pi_8(t)$ on $[0,8]$.

Theorems & Definitions (34)

• Theorem 2.1: Whittaker-Kotel’nikov-Shannon sampling theorem
• Definition 2.1: Bandlimited functions on $\mathbb{R}^d$
• Theorem 2.2: Parzen sampling theorem
• Lemma 2.1
• Theorem 3.1: Reconstruction from regularized sampling
• Theorem 3.2: Generalization in dimension one using proxy GNN \ref{['proxygnnlayer']}
• Theorem 3.3
• Theorem 3.4: Generalization using GNN
• Theorem 3.5
• Remark 3.1