REGE: A Method for Incorporating Uncertainty in Graph Embeddings

TL;DR

REGE tackles uncertainty in graph embeddings by separating data-induced uncertainty (via eigen-decomposition to create multiple graph views and a consensus edge-uncertainty matrix) from model-induced uncertainty (via conformalized, per-dimension quantile regression in a teacher-student setup). It then integrates these radii into training by injecting noise into hidden representations and employing curriculum learning over progressively richer graph views. Empirical results on Cora, Citeseer, and PolBlogs show that REGE improves adversarial robustness by about 1.5% in node classification accuracy compared to state-of-the-art defenses. The work provides a principled framework for uncertainty-aware graph embeddings with practical implications for robust graph-based learning systems.

Abstract

Machine learning models for graphs in real-world applications are prone to two primary types of uncertainty: (1) those that arise from incomplete and noisy data and (2) those that arise from uncertainty of the model in its output. These sources of uncertainty are not mutually exclusive. Additionally, models are susceptible to targeted adversarial attacks, which exacerbate both of these uncertainties. In this work, we introduce Radius Enhanced Graph Embeddings (REGE), an approach that measures and incorporates uncertainty in data to produce graph embeddings with radius values that represent the uncertainty of the model's output. REGE employs curriculum learning to incorporate data uncertainty and conformal learning to address the uncertainty in the model's output. In our experiments, we show that REGE's graph embeddings perform better under adversarial attacks by an average of 1.5% (accuracy) against state-of-the-art methods.

Figures (5)

• Figure 1: REGE uses eigen-decomposition to generate multiple graph views by systematically reconstructing the graph with an increasing number of components. These views are used to compute data-dependent radii (DDR) using a consensus and binary deviation function. A student-teacher model with quantile loss is used to capture uncertainty around each embedding dimension for each node to compute model-dependent radii (MDR). DDR or MDR is then incorporated into the training procedure as noise that is injected into the hidden layer representations. The generated graph views are used to train the graph embedding function using curriculum learning. Doing so leads to embeddings with higher accuracy under adversarial attacks on node classification.
• Figure 2: Radii for each node in the PolBlogs network, derived using binary deviation, standard deviation, and entropy functions, plotted against node degree. Unlike standard deviation and entropy, which correlate radii closely with degree, the binary deviation function allows low-degree nodes to also have low radii, capturing more information than just degree distribution. (inset) The binary deviation function calculates uncertainties based on entries of the weighted adjacency matrix $W$. For an edge between nodes $i$ and $j$, $W_{ij} = 0$ indicates that the edge was never observed, while $W_{ij} = 1$ indicates the edge was always observed, both representing high certainty. In contrast, $W_{ij} = 0.5$ corresponds to the highest level of uncertainty.
• Figure 5: Low degree nodes (bold) have low DDR possibly due to consistent edge reconstruction but high MDR indicating graph embedding functions may struggle to learn robust representations for low degree nodes.
• Figure 6: (a) Visualization of the Karate Club network with (b) data-dependent and (c) model-dependent radii. In (b), low-degree nodes (e.g., 11, 12, 17, 18) exhibit small radii, indicating high confidence due to consistent edge reconstruction. In (c), nodes on the network's periphery show larger model-dependent radii, reflecting uncertainty in the learning algorithm, especially for low-degree nodes.
• Figure 7: Comparison between REGE and GCN-SVD under varying numbers of components for datasets perturbed by Meta Attack at a 10% perturbation rate. REGE consistently demonstrates higher accuracy across all datasets with fewer components.