Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HybSpecNet: A Critical Analysis of Architectural Instability in Hybrid-Domain Spectral GNNs

Huseyin Goksu

TL;DR

This work addresses the stability-adaptivity trade-off in spectral graph neural networks by proposing HybSpecNet, a hybrid-domain architecture that combines a stable ChebyNet branch in the $[-1,1]$ spectrum with an adaptive KrawtchoukNet branch in the $[0,\infty)$ spectrum. It identifies a critical pitfall called Instability Poisoning in naive fusion hybrids, where unstable gradients from the adaptive branch can derail the whole model, as demonstrated by HybSpecNet-v3 at high $K$. The authors propose HybSpecNet-v4 with a Late Fusion design that isolates gradient pathways, achieving numerical stability up to $K=30$ while maintaining state-of-the-art performance across homophilic and heterophilic graphs. The work offers a practical architectural blueprint for robust hybrid-domain GNNs and highlights a key consideration for future adaptive, deep spectral models. Overall, HybSpecNet advances reliable, unified performance on diverse graph types by decoupling gradient flow between spectral domains.

Abstract

Spectral Graph Neural Networks offer a principled approach to graph filtering but face a fundamental "Stability-vs-Adaptivity" trade-off. This trade-off is dictated by the choice of spectral domain. Filters in the finite [-1, 1] domain (e.g., ChebyNet) are numerically stable at high polynomial degrees (K) but are static and low-pass, causing them to fail on heterophilic graphs. Conversely, filters in the semi-infinite [0, infty) domain (e.g., KrawtchoukNet) are highly adaptive and achieve SOTA results on heterophily by learning non-low-pass responses. However, as we demonstrate, these adaptive filters can also suffer from numerical instability, leading to catastrophic performance collapse at high K. In this paper, we propose to resolve this trade-off by designing a hybrid-domain GNN, HybSpecNet, which combines a stable `ChebyNet` branch with an adaptive `KrawtchoukNet` branch. We first demonstrate that a "naive" hybrid architecture, which fuses the branches via concatenation, successfully unifies performance at low K, achieving strong results on both homophilic and heterophilic benchmarks. However, we then prove that this naive architecture fails the stability test. Our K-ablation experiments show that this architecture catastrophically collapses at K=25, exactly mirroring the collapse of its unstable `KrawtchoukNet` branch. We identify this critical finding as "Instability Poisoning," where `NaN`/`Inf` gradients from the adaptive branch destroy the training of the model. Finally, we propose and validate an advanced architecture that uses "Late Fusion" to completely isolate the gradient pathways. We demonstrate that this successfully solves the instability problem, remaining perfectly stable up to K=30 while retaining its SOTA performance across all graph types. This work identifies a critical architectural pitfall in hybrid GNN design and provides the robust architectural solution.

HybSpecNet: A Critical Analysis of Architectural Instability in Hybrid-Domain Spectral GNNs

TL;DR

This work addresses the stability-adaptivity trade-off in spectral graph neural networks by proposing HybSpecNet, a hybrid-domain architecture that combines a stable ChebyNet branch in the spectrum with an adaptive KrawtchoukNet branch in the spectrum. It identifies a critical pitfall called Instability Poisoning in naive fusion hybrids, where unstable gradients from the adaptive branch can derail the whole model, as demonstrated by HybSpecNet-v3 at high . The authors propose HybSpecNet-v4 with a Late Fusion design that isolates gradient pathways, achieving numerical stability up to while maintaining state-of-the-art performance across homophilic and heterophilic graphs. The work offers a practical architectural blueprint for robust hybrid-domain GNNs and highlights a key consideration for future adaptive, deep spectral models. Overall, HybSpecNet advances reliable, unified performance on diverse graph types by decoupling gradient flow between spectral domains.

Abstract

Spectral Graph Neural Networks offer a principled approach to graph filtering but face a fundamental "Stability-vs-Adaptivity" trade-off. This trade-off is dictated by the choice of spectral domain. Filters in the finite [-1, 1] domain (e.g., ChebyNet) are numerically stable at high polynomial degrees (K) but are static and low-pass, causing them to fail on heterophilic graphs. Conversely, filters in the semi-infinite [0, infty) domain (e.g., KrawtchoukNet) are highly adaptive and achieve SOTA results on heterophily by learning non-low-pass responses. However, as we demonstrate, these adaptive filters can also suffer from numerical instability, leading to catastrophic performance collapse at high K. In this paper, we propose to resolve this trade-off by designing a hybrid-domain GNN, HybSpecNet, which combines a stable `ChebyNet` branch with an adaptive `KrawtchoukNet` branch. We first demonstrate that a "naive" hybrid architecture, which fuses the branches via concatenation, successfully unifies performance at low K, achieving strong results on both homophilic and heterophilic benchmarks. However, we then prove that this naive architecture fails the stability test. Our K-ablation experiments show that this architecture catastrophically collapses at K=25, exactly mirroring the collapse of its unstable `KrawtchoukNet` branch. We identify this critical finding as "Instability Poisoning," where `NaN`/`Inf` gradients from the adaptive branch destroy the training of the model. Finally, we propose and validate an advanced architecture that uses "Late Fusion" to completely isolate the gradient pathways. We demonstrate that this successfully solves the instability problem, remaining perfectly stable up to K=30 while retaining its SOTA performance across all graph types. This work identifies a critical architectural pitfall in hybrid GNN design and provides the robust architectural solution.

Paper Structure

This paper contains 17 sections, 6 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Paradigm 1: Architectural Solutions (Stability-Focused)
  4. Paradigm 2: Coefficient-Learning (Flexibility-Focused)
  5. Paradigm 3: Basis-Learning (AOPF Class)
  6. The Need for a Hybrid Architecture
  7. Methodology
  8. The Spectral Domain Trade-off
  9. Filter Components
  10. Architecture 1: HybSpecNet-v3 (The Naive Concatenation Pitfall)
  11. Architecture 2: HybSpecNet-v4 (The Solution: Late Fusion)
  12. Architecture 2: HybSpecNet-v4 (The Solution: Late Fusion)
  13. Experiments
  14. Experimental Setup
  15. H1 Results: Unified Performance ($K=3$)
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Comparison of hybrid architectures. (Left) 'HybSpecNet-v3' (Naive Concatenation). Branches (Het and Stab) are fused at each layer. This allows 'NaN' gradients (red arrow) from the 'KrawtchoukNet' branch at $K=25$ to "poison" the stable 'ChebConv' branch. (Right) 'HybSpecNet-v4' (Late Fusion). The models run in parallel, and only their final predictions ($log\_softmax$) are averaged. Gradient pathways are isolated, and the collapse of one branch does not affect the other.
  • Figure 2: $K$ (Polynomial Degree) vs. Test Accuracy (PubMed). This plot visualizes our core finding from Table \ref{['tab:h2_stability']}. The 'KrawtchoukNet' (green) filter collapses at $K=25$. The naive 'HybSpecNet-v3' (red) inherits this collapse. Our proposed solution, 'HybSpecNet-v4' (cyan), isolates the gradients and successfully remains stable, tracking the robust 'ChebyNet' (blue) baseline.