Graph Structure Refinement with Energy-based Contrastive Learning

TL;DR

The paper tackles the vulnerability of Graph Neural Networks to noisy or missing graph structure by introducing Energy-based Contrastive Learning (ECL) to guide Graph Structure Refinement (GSR). It unifies an energy-based generative term with a contrastive discriminative term, forming a joint objective that learns improved node representations while refining topology. The approach constructs a dual-attribute graph, learns representations via ECL with SGLD-based sampling, and predicts a refined adjacency to feed downstream node classification, achieving state-of-the-art results on eight benchmarks with better efficiency and robustness. The methodology demonstrates strong empirical gains, ablation-backed evidence of effectiveness, and robustness to structural perturbations, indicating practical potential for reliable graph learning in real-world noisy settings.

Abstract

Graph Neural Networks (GNNs) have recently gained widespread attention as a successful tool for analyzing graph-structured data. However, imperfect graph structure with noisy links lacks enough robustness and may damage graph representations, therefore limiting the GNNs' performance in practical tasks. Moreover, existing generative architectures fail to fit discriminative graph-related tasks. To tackle these issues, we introduce an unsupervised method based on a joint of generative training and discriminative training to learn graph structure and representation, aiming to improve the discriminative performance of generative models. We propose an Energy-based Contrastive Learning (ECL) guided Graph Structure Refinement (GSR) framework, denoted as ECL-GSR. To our knowledge, this is the first work to combine energy-based models with contrastive learning for GSR. Specifically, we leverage ECL to approximate the joint distribution of sample pairs, which increases the similarity between representations of positive pairs while reducing the similarity between negative ones. Refined structure is produced by augmenting and removing edges according to the similarity metrics among node representations. Extensive experiments demonstrate that ECL-GSR outperforms the state-of-the-art on eight benchmark datasets in node classification. ECL-GSR achieves faster training with fewer samples and memories against the leading baseline, highlighting its simplicity and efficiency in downstream tasks.

Figures (9)

• Figure 1: Illustration of the procedure of ECL-GSR. Preprocessed dual-attribute graph undergoes data augmentations, energy-based contrastive learning, and edge prediction, achieving structure refinement. Refined graph is applied in node classification.
• Figure 2: Training time and space analysis on Cora, Citeseer, Actor, and Pubmed datasets.
• Figure 3: Performance study of ECL-GSR variants and other baselines over multiple training epochs on four datasets.
• Figure 4: Hyperparameter $\mu$ and dimensionality $\widetilde{F}$ analysis of ECL-GSR on four datasets. “Mean” denotes the averages.
• Figure 5: Hyperparameter $\alpha$ and dimensionality $\widetilde{F}$ analysis of ECL-GSR on four datasets.