Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily

TL;DR

This work tackles spectral mismatch in graph pre-training and prompt tuning across graphs with varying homophily. It introduces HS-GPPT, which combines a hybrid spectral backbone (Beta Wavelet GNN) with spectral-aligned prompt tuning—training prompts per pre-trained filter to reshape downstream spectra toward the backbone’s diverse frequency bands. Theoretical results establish the Spectral Specificity principle and demonstrate spectral distribution diversity tied to homophily, motivating abundant spectral knowledge and adaptive prompts. Empirical results show HS-GPPT achieving superior performance, especially on heterophilic graphs, under both transductive and inductive settings, with ablations validating the necessity of hybrid filters and per-filter prompts. The approach offers a principled, scalable path for robust knowledge transfer in spectrum-diverse graphs.

Abstract

Graph ``pre-training and prompt-tuning'' aligns downstream tasks with pre-trained objectives to enable efficient knowledge transfer under limited supervision. However, current methods typically rely on single-filter backbones (e.g., low-pass), whereas real-world graphs exhibit inherent spectral diversity. Our theoretical \textit{Spectral Specificity} principle reveals that effective knowledge transfer requires alignment between pre-trained spectral filters and the intrinsic spectrum of downstream graphs. This identifies two fundamental limitations: (1) Knowledge Bottleneck: single-filter models suffer from irreversible information loss by suppressing signals from other frequency bands (e.g., high-frequency); (2) Utilization Bottleneck: spectral mismatches between pre-trained filters and downstream spectra lead to significant underutilization of pre-trained knowledge. To bridge this gap, we propose HS-GPPT. We utilize a hybrid spectral backbone to construct an abundant knowledge basis. Crucially, we introduce Spectral-Aligned Prompt Tuning to actively align the downstream graph's spectrum with diverse pre-trained filters, facilitating comprehensive knowledge utilization across both homophily and heterophily. Extensive experiments validate the effectiveness under both transductive and inductive learning settings.

Figures (8)

• Figure 1: Distribution of $S_{high}$ (high-frequency area) across different feature dimensions in various datasets.
• Figure 2: The overall framework of our HS-GPPT. In the pre-training stage, the graph filters and integration weights are trained. In the prompt tuning stage, we keep the graph filters and integration weights frozen and only tune the learnable prompt graphs and task head (i.e., one-layer MLP).
• Figure 3: Compatibility investigation with different hybrid GNN backbones. Here 'best baseline' denotes the best baseline results.
• Figure 4: Integration weights of filters: $\overline{\mathbf{x}^0}$, $\overline{\mathbf{x}^1}$, $\overline{\mathbf{x}^2}$ correspond to $g_{0,2}$ (low-pass), $g_{1,1}$ (band-pass), and $g_{2,0}$ (low-pass).
• Figure 5: Case studies on graphs with varying homophily levels. The left figure illustrates the spectrum $g(\mathbf{\Lambda})$ of different filters, while the right figure shows the performance on graphs with homophily levels from 0 to 1.

Theorems & Definitions (7)

• Definition 3.1: High-frequency Area
• Proposition 3.2: Homophily-Spectrum Correlation
• Theorem 3.3: Spectral Specificity
• Theorem 4.1: Spectral Adaptability
• Corollary 4.2: Spectral Alignment
• Proposition A.1
• Lemma A.2