Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learning

Zixu Wang, Bingbing Xu, Yige Yuan, Huawei Shen, Xueqi Cheng

TL;DR

This work tackles the limited OOD generalization of Graph Contrastive Learning by identifying that treating cross-domain pairs solely as negatives in InfoNCE exacerbates domain gaps. It introduces Negative as Positive (NaP), a framework that reclassifies the most semantically similar cross-domain negatives as positives, implemented via an Encoding Module and a two-stage Objective Module (Warm-Up and NaP stages) with a new loss term. Empirical evaluations on the GOOD benchmark and Facebook100 show that NaP substantially improves OOD performance, narrows the Pairwise Domain Discrepancy, and yields semantically aligned cross-domain pairs, outperforming multiple InfoNCE-based baselines. The findings highlight a practical strategy to enhance cross-domain transfer in graph pre-training, with potential implications for other contrastive learning settings.

Abstract

Graph contrastive learning (GCL), standing as the dominant paradigm in the realm of graph pre-training, has yielded considerable progress. Nonetheless, its capacity for out-of-distribution (OOD) generalization has been relatively underexplored. In this work, we point out that the traditional optimization of InfoNCE in GCL restricts the cross-domain pairs only to be negative samples, which inevitably enlarges the distribution gap between different domains. This violates the requirement of domain invariance under OOD scenario and consequently impairs the model's OOD generalization performance. To address this issue, we propose a novel strategy "Negative as Positive", where the most semantically similar cross-domain negative pairs are treated as positive during GCL. Our experimental results, spanning a wide array of datasets, confirm that this method substantially improves the OOD generalization performance of GCL.

Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learning

TL;DR

This work tackles the limited OOD generalization of Graph Contrastive Learning by identifying that treating cross-domain pairs solely as negatives in InfoNCE exacerbates domain gaps. It introduces Negative as Positive (NaP), a framework that reclassifies the most semantically similar cross-domain negatives as positives, implemented via an Encoding Module and a two-stage Objective Module (Warm-Up and NaP stages) with a new loss term. Empirical evaluations on the GOOD benchmark and Facebook100 show that NaP substantially improves OOD performance, narrows the Pairwise Domain Discrepancy, and yields semantically aligned cross-domain pairs, outperforming multiple InfoNCE-based baselines. The findings highlight a practical strategy to enhance cross-domain transfer in graph pre-training, with potential implications for other contrastive learning settings.

Abstract

Graph contrastive learning (GCL), standing as the dominant paradigm in the realm of graph pre-training, has yielded considerable progress. Nonetheless, its capacity for out-of-distribution (OOD) generalization has been relatively underexplored. In this work, we point out that the traditional optimization of InfoNCE in GCL restricts the cross-domain pairs only to be negative samples, which inevitably enlarges the distribution gap between different domains. This violates the requirement of domain invariance under OOD scenario and consequently impairs the model's OOD generalization performance. To address this issue, we propose a novel strategy "Negative as Positive", where the most semantically similar cross-domain negative pairs are treated as positive during GCL. Our experimental results, spanning a wide array of datasets, confirm that this method substantially improves the OOD generalization performance of GCL.
Paper Structure (23 sections, 10 equations, 5 figures, 2 tables)

This paper contains 23 sections, 10 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Task Formulation of OOD in GCL
  4. Pairwise Domain Discrepancy
  5. Proposed Method
  6. Motivation
  7. NaP: Negative as Positive
  8. Encoding Module
  9. Objective Module
  10. (1) Warm-Up Stage:
  11. (2) NaP Stage:
  12. Experiments
  13. Datasets
  14. Experimental Setup
  15. Data settings
  16. ...and 8 more sections

Figures (5)

  • Figure 1: Left: Traditional GCLs perform badly under OOD scenario compared to IID one. Right: Pairwize-Domain-Discrepancy grows during GCL.
  • Figure 2: Left: All CDPs are negative samples. Right: PDD decreases while more CDPs are removed.
  • Figure 3: The overall framework of NaP consists of two modules: the encoding module and the objective module. The objective module comprises two stages: the warm-up stage and the NaP stage.
  • Figure 4: Experiments result of NaP and GRACE on 10 OOD target domains from Facebook100.
  • Figure 5: t-SNE visualization and PDD of node embedding.