Training Robust Graph Neural Networks by Modeling Noise Dependencies

TL;DR

DA-GNN tackles robustness of GNNs under a realistic dependency-aware noise on graphs (DANG), where node feature noise propagates to both graph structure and node labels. It introduces a variational deep generative framework that jointly learns latent clean graph structure $Z_A$, latent clean labels $Z_Y$, and noise indicators $\epsilon$ by maximizing the ELBO of $P(X,A,Y)$. The approach demonstrates superior node classification and link prediction robustness on both synthetic DANG and real-world Auto/Garden benchmarks, while remaining competitive under conventional noise models. By directly modeling the causal data-generating process, the work narrows the gap between practical noise patterns and robust graph learning, enabling more reliable deployment in real applications.

Abstract

In real-world applications, node features in graphs often contain noise from various sources, leading to significant performance degradation in GNNs. Although several methods have been developed to enhance robustness, they rely on the unrealistic assumption that noise in node features is independent of the graph structure and node labels, thereby limiting their applicability. To this end, we introduce a more realistic noise scenario, dependency-aware noise on graphs (DANG), where noise in node features create a chain of noise dependencies that propagates to the graph structure and node labels. We propose a novel robust GNN, DA-GNN, which captures the causal relationships among variables in the data generating process (DGP) of DANG using variational inference. In addition, we present new benchmark datasets that simulate DANG in real-world applications, enabling more practical research on robust GNNs. Extensive experiments demonstrate that DA-GNN consistently outperforms existing baselines across various noise scenarios, including both DANG and conventional noise models commonly considered in this field. Our code is available at https://github.com/yeonjun-in/torch-DA-GNN.

Figures (14)

• Figure 1: Examples of DANG in social networks: IFN represents independent node feature noise. Under the IFN (b), Bob’s noisy features have no effect on the graph structure or node labels. However, in DANG (c), Bob’s noisy features can propagate, leading to both structural noise in the graph and label noise.
• Figure 2: A directed graphical model indicating a DGP of (a) IFN, and (b) DANG.
• Figure 3: Overall architecture of DA-GNN. (a) With the noisy graph $(X,A,Y)$ as inputs, we design the inference encoders ($\phi_1$, $\phi_2$ and $\phi_3$) and regularizers ($\mathcal{L}_{\text{hom}}$, $\mathcal{L}_{\text{cls-enc}}$, and $\mathcal{L}_{\text{p}}$) to infer $Z_A$, $Z_Y$, $\epsilon_A$, and $\epsilon_X$. (b) Leveraging the inferred latent variables, we formulate the generative decoders ($\theta_1$, $\theta_2$, and $\theta_3$) and reconstruction loss functions ($\mathcal{L}_{\text{rec-edge}}$, $\mathcal{L}_{\text{rec-feat}}$, and $\mathcal{L}_{\text{cls-dec}}$) to capture the causal relationships that generate noise in the graph.
• Figure 3: Ablation studies of various DGPs from Fig \ref{['fig:ablation_dgp']}. Case 3 removes $Y \leftarrow (X,A)$; Case 2 additionally removes $A \leftarrow X$; Case 1 additionally removes $A \leftarrow \epsilon$, equivalent to IFN (Fig \ref{['fig:DANG_graphical']}).
• Figure 4: Node classification under node feature noise, structure noise, and node label noise scenarios, which are commonly considered in robust GNN research field, on Cora dataset.