Sparse Bayesian Message Passing under Structural Uncertainty

TL;DR

The paper tackles semi-supervised node classification on graphs with structural noise and heterophily. It introduces SpaM, a Sparse Bayesian Message Passing framework, that maintains a posterior over signed adjacencies $Z \in \{-1,0,+1\}^{n \times n}$ and marginalizes predictions via Monte Carlo sampling, using a local sparse coding layer for aggregation. Theoretical contributions include a risk decomposition showing excess risk scales with the posterior approximation error, and empirical results across nine heterophilic benchmarks demonstrating improved robustness and accuracy over strong baselines. This uncertainty-aware graph learning framework provides a principled alternative to fixed-graph or purely parameter-based uncertainty methods and scales to large, noisy graphs.

Abstract

Semi-supervised learning on real-world graphs is frequently challenged by heterophily, where the observed graph is unreliable or label-disassortative. Many existing graph neural networks either rely on a fixed adjacency structure or attempt to handle structural noise through regularization. In this work, we explicitly capture structural uncertainty by modeling a posterior distribution over signed adjacency matrices, allowing each edge to be positive, negative, or absent. We propose a sparse signed message passing network that is naturally robust to edge noise and heterophily, which can be interpreted from a Bayesian perspective. By combining (i) posterior marginalization over signed graph structures with (ii) sparse signed message aggregation, our approach offers a principled way to handle both edge noise and heterophily. Experimental results demonstrate that our method outperforms strong baseline models on heterophilic benchmarks under both synthetic and real-world structural noise.

Figures (7)

• Figure 1: Architecture of the Sparse Bayesian Message Passing Network (SpaM), which consists of three main modules: (i) Structural Uncertainty & Sampling (Block 1): A Variational Graph Autoencoder (VGAE) learns the posterior $q_\phi(Z \mid A_{\text{obs}}, X, \ldots)$ over the latent signed graph $Z$. (ii) S$^2$ Layer (Block 2): For a sampled $Z$, each node $i$ solves a local LASSO problem $\min_\alpha \lVert t_i - V_i \alpha \rVert_2^2 + \lambda \lVert \alpha \rVert_1$ to find a sparse coefficient vector $\alpha_i^\ast$ from its neighbors. (iii) Prediction & Joint Training (Block 3): The S$^2$ layers are stacked $L$ times, and predictions $p_\theta(y_i \mid X, Z^{(k)})$ from all $K$ samples are averaged to form the final predictive distribution (Monte Carlo averaging).
• Figure 2: (Q2) Structural posterior modeling on the Texas dataset. (a) Edge-wise posterior distributions over signed relations $q_\phi(z_{ij})$. (b) Accuracy comparison of ablation variants.
• Figure 3: (Q3) Effect of sparse signed message passing on depth robustness (Texas dataset). (a) Accuracy as network depth increases. (b) Oversmoothing behavior under ablations removing sparsity or sign-aware aggregation.
• Figure 4: (Q4) Robustness under three perturbations on the Texas dataset: (a) random edge deletions, (b) additive Gaussian feature noise, and (c) adversarial edge perturbations.
• Figure 5: Illustrative comparison of baseline GNNs and SpaM. Left: A baseline GNN aggregates messages from all observed neighbors of node $0$ in $\mathcal{G}_{\text{obs}}$, treating them as equally informative. Right: SpaM first infers a posterior over signed edges $q_\phi(Z \mid A_{\text{obs}}, X, Y_{\mathcal{L}})$ and converts raw edges into Bayesian messages $\{z_{0j} \in \{-1,0,+1\}\}$, indicating positive, negative, or absent relations.

Theorems & Definitions (4)

• Theorem 5.1: Risk decomposition under structural approximation
• Theorem D.1: Posterior consistency of signed edges
• Theorem D.2: Signed aggregation enlarges cluster margin
• Theorem D.3: Support recovery of sparse coding under CSBM