Joint Graph Rewiring and Feature Denoising via Spectral Resonance

TL;DR

The paper tackles the problem of noisy graph structure and node features impeding downstream node classification. It introduces Joint Denoising and Rewiring (JDR), an alternating spectral interpolation approach that maximizes the alignment between the leading graph eigenvectors and feature singular vectors, defined by $Alignment_L(A,X) = ||V_L^T U_L||_{sp}$, to achieve spectral resonance. The authors provide theoretical support under stylized GOE-like noise and demonstrate empirical superiority over existing preprocessing rewiring methods on both synthetic contextual SBMs and real-world datasets across homophilic and heterophilic regimes. The method offers a practical preprocessing tool that enhances GNN performance by leveraging the joint information in graphs and features, while also acknowledging limitations such as the necessity of node features and potential avenues for combining with other rewiring strategies.

Abstract

When learning from graph data, the graph and the node features both give noisy information about the node labels. In this paper we propose an algorithm to jointly denoise the features and rewire the graph (JDR), which improves the performance of downstream node classification graph neural nets (GNNs). JDR works by aligning the leading spectral spaces of graph and feature matrices. It approximately solves the associated non-convex optimization problem in a way that handles graphs with multiple classes and different levels of homophily or heterophily. We theoretically justify JDR in a stylized setting and show that it consistently outperforms existing rewiring methods on a wide range of synthetic and real-world node classification tasks.

Figures (19)

• Figure 1: Schematic overview of jdr. In this example, we consider a noisy graph as it occurs in many different real-world scenarios, in the sense that it contains edges between and within classes and its node features are not fully aligned with the labels. The graph's adjacency matrix $\bm{A}$ and binary node features $\bm{X}$ are decomposed via spectral decomposition and . The rewiring of $\bm{A}$ is performed by combining the information of its own eigenvectors $\bm{V}$ and the singular vectors $\bm{U}$ from $\bm{X}$. The same applies vice versa for denoising, and both are performed iteratively $K$ times. We synthesize the rewired graph $\tilde{\bm{A}}$ and the denoised features $\tilde{\bm{X}}$ by multiplying back with the final $\bm{V}_{(K)}$ and $\bm{U}_{(K)}$. To get specific properties like sparsity or binarity we can perform an Update step, e.g. by thresholding (as done here). The resulting denoised and rewired graph is displayed on the right. Its structure now better represents the communities and the first entry of the features indicates the class assignment.
• Figure 2: An illustration of spectral alignment and resonance. In (\ref{['fig:graph_graph']}) we plot $r=\bm{x}^T\bm{A}\bm{x}$ for different noise levels in $\bm{A}$ and $\bm{x} \in \{-1, 1\}^N$, illustrated in the rows below. Without noise, $\bm{x}$ is exactly the label vector and $\bm{A}$ is block-diagonal. We apply multiplicative noise; namely, for each noise level, we flip the sign of a proportion of values, resulting in a random signal for $\pm 0.5$. We see that the value of $r$ depends on the noise level. The maximum is achieved for zero noise when the second leading eigenvector of $\bm{A}$ and the signal $\bm{x}$ are perfectly aligned. In (\ref{['fig:graph_4x4']}), we consider a signal $\bm{\hat{x}}=\bm{A}\bm{x}$ for different noise levels in $\bm{A}$ and $\bm{x}$ on a graph with $20$ nodes; only a quarter of edges are shown to reduce clutter; the intra-class edges are grey; the inter-class edges are black. The largest norm is obtained for noise-free $\bm{A}$ and $\bm{x}$ (upper-left corner). The norm of $\hat{\bm{x}}$ and the separation of communities decrease along both noise axes. The inherent denoising capabilities of propagating $\bm{x}$ on a high- graph ma2021unified are also visible, particularly in the first two rows to the right.
• Figure 3: Alignment of the leading eigenspaces according to \ref{['eq: alignment']} for graphs from the with different $\phi$.
• Figure 4: Test accuracy on graphs from the across different $\phi$. The error bars indicate the $95\%$ confidence interval. improves the performance for both across all $\phi$.
• Figure 5: Visualization of the first six eigenmodes of $\bm{L}$ of the $8 \times 8$ grid graph.

Theorems & Definitions (3)

• Definition 1
• Proposition 1
• proof