Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contrastive Learning Meets Pseudo-label-assisted Mixup Augmentation: A Comprehensive Graph Representation Framework from Local to Global

Jinlu Wang, Yanfeng Sun, Jiapu Wang, Junbin Gao, Shaofan Wang, Jipeng Guo

TL;DR

The paper addresses the limited global context in traditional GNNs by introducing ComGRL, a framework that fuses local graph contrastive learning (LGCL) with global multi-head self-attention (GMSA) and ties them together with pseudo-label-assisted Mixup augmentation (PMA). LGCL implicitly smooths local structure and feeds enhanced representations into GMSA, which expands to diverse global correlations; PMA leverages high-confidence pseudo-labels to perform Mixup across attributes and structure, fostering mutual reinforcement between local and global modules. A two-stage training scheme—pre-training to obtain pseudo-labels followed by fine-tuning with Mixup—achieves state-of-the-art node classification across six datasets and exhibits robustness to graph and label noise. The approach broadens the receptive field and offers a practical, self-supervised mechanism to integrate local and global information for comprehensive graph representations with strong empirical performance and resilience.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable effectiveness in various graph representation learning tasks. However, most existing GNNs focus primarily on capturing local information through explicit graph convolution, often neglecting global message-passing. This limitation hinders the establishment of a collaborative interaction between global and local information, which is crucial for comprehensively understanding graph data. To address these challenges, we propose a novel framework called Comprehensive Graph Representation Learning (ComGRL). ComGRL integrates local information into global information to derive powerful representations. It achieves this by implicitly smoothing local information through flexible graph contrastive learning, ensuring reliable representations for subsequent global exploration. Then ComGRL transfers the locally derived representations to a multi-head self-attention module, enhancing their discriminative ability by uncovering diverse and rich global correlations. To further optimize local information dynamically under the self-supervision of pseudo-labels, ComGRL employs a triple sampling strategy to construct mixed node pairs and applies reliable Mixup augmentation across attributes and structure for local contrastive learning. This approach broadens the receptive field and facilitates coordination between local and global representation learning, enabling them to reinforce each other. Experimental results across six widely used graph datasets demonstrate that ComGRL achieves excellent performance in node classification tasks. The code could be available at https://github.com/JinluWang1002/ComGRL.

Contrastive Learning Meets Pseudo-label-assisted Mixup Augmentation: A Comprehensive Graph Representation Framework from Local to Global

TL;DR

The paper addresses the limited global context in traditional GNNs by introducing ComGRL, a framework that fuses local graph contrastive learning (LGCL) with global multi-head self-attention (GMSA) and ties them together with pseudo-label-assisted Mixup augmentation (PMA). LGCL implicitly smooths local structure and feeds enhanced representations into GMSA, which expands to diverse global correlations; PMA leverages high-confidence pseudo-labels to perform Mixup across attributes and structure, fostering mutual reinforcement between local and global modules. A two-stage training scheme—pre-training to obtain pseudo-labels followed by fine-tuning with Mixup—achieves state-of-the-art node classification across six datasets and exhibits robustness to graph and label noise. The approach broadens the receptive field and offers a practical, self-supervised mechanism to integrate local and global information for comprehensive graph representations with strong empirical performance and resilience.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable effectiveness in various graph representation learning tasks. However, most existing GNNs focus primarily on capturing local information through explicit graph convolution, often neglecting global message-passing. This limitation hinders the establishment of a collaborative interaction between global and local information, which is crucial for comprehensively understanding graph data. To address these challenges, we propose a novel framework called Comprehensive Graph Representation Learning (ComGRL). ComGRL integrates local information into global information to derive powerful representations. It achieves this by implicitly smoothing local information through flexible graph contrastive learning, ensuring reliable representations for subsequent global exploration. Then ComGRL transfers the locally derived representations to a multi-head self-attention module, enhancing their discriminative ability by uncovering diverse and rich global correlations. To further optimize local information dynamically under the self-supervision of pseudo-labels, ComGRL employs a triple sampling strategy to construct mixed node pairs and applies reliable Mixup augmentation across attributes and structure for local contrastive learning. This approach broadens the receptive field and facilitates coordination between local and global representation learning, enabling them to reinforce each other. Experimental results across six widely used graph datasets demonstrate that ComGRL achieves excellent performance in node classification tasks. The code could be available at https://github.com/JinluWang1002/ComGRL.

Paper Structure

This paper contains 17 sections, 20 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations
  4. Mixup Augmentation
  5. Methodology
  6. Local Graph Contrastive Learning
  7. Global Multi-head Self-attention
  8. Pseudo-label-assisted Mixup Augmentation
  9. Training Process
  10. Experiments
  11. Datasets
  12. Experimental Settings
  13. Baselines
  14. Experimental Results and Analysis
  15. Ablation Analysis
  16. ...and 2 more sections

Figures (2)

  • Figure 1: The overall framework of the proposed ComGRL method primarily consists of the LGCL, GMSA, and PMA modules. In the first pre-training process, the LGCL and GMSA modules are unified to learn comprehensive representation and generate global, relatively reliable pseudo labels. In the second fine-tuning process, using the global pseudo-label information, the PMA module adaptively conducts the node Mixup augmentation for the LGCL module, achieving the interaction between local and global information in a self-supervised manner.
  • Figure 2: The experimental results across different ablation settings in PMA module.