Subgraph Networks Based Contrastive Learning

TL;DR

This work introduces SubGraph Network-based Contrastive Learning (SGNCL), a framework that leverages Edge-To-Node derived subgraph networks to capture high-order interactions among substructures for graph contrastive learning. By employing separate encoders for first-order and second-order SGNs and a fused contrastive objective, SGNCL improves both unsupervised graph representations and transfer learning performance on molecular and network datasets. Empirical results show competitive unsupervised performance and a 6.9% average gain in ROC-AUC on transfer tasks, underscoring the value of modeling substructure interactions. The approach offers a scalable, structure-aware augmentation paradigm that enhances GNN representations across diverse graph domains.

Abstract

Graph contrastive learning (GCL), as a self-supervised learning method, can solve the problem of annotated data scarcity. It mines explicit features in unannotated graphs to generate favorable graph representations for downstream tasks. Most existing GCL methods focus on the design of graph augmentation strategies and mutual information estimation operations. Graph augmentation produces augmented views by graph perturbations. These views preserve a locally similar structure and exploit explicit features. However, these methods have not considered the interaction existing in subgraphs. To explore the impact of substructure interactions on graph representations, we propose a novel framework called subgraph network-based contrastive learning (SGNCL). SGNCL applies a subgraph network generation strategy to produce augmented views. This strategy converts the original graph into an Edge-to-Node mapping network with both topological and attribute features. The single-shot augmented view is a first-order subgraph network that mines the interaction between nodes, node-edge, and edges. In addition, we also investigate the impact of the second-order subgraph augmentation on mining graph structure interactions, and further, propose a contrastive objective that fuses the first-order and second-order subgraph information. We compare SGNCL with classical and state-of-the-art graph contrastive learning methods on multiple benchmark datasets of different domains. Extensive experiments show that SGNCL achieves competitive or better performance (top three) on all datasets in unsupervised learning settings. Furthermore, SGNCL achieves the best average gain of 6.9\% in transfer learning compared to the best method. Finally, experiments also demonstrate that mining substructure interactions have positive implications for graph contrastive learning.

Figures (6)

• Figure 1: Fundamentals of Subgraph network (SGN) transformation. Specifically, the edges in the original graph are transformed into nodes in SGNs, and the adjacent edges sharing the same node in the original graph will be connected in the SGN. We subsequently extend the rules of SGN and use it as an augmented view.
• Figure 2: A general framework for SGNCL. In the graph augmentation module, we use an SGN-based strategy to generate subgraph networks (SGNs) in an Edge-to-node fashion with topological and attribute features. Specifically, the first-order SGN mines high-order interaction information in node-node or node-edge; The second-order SGN mines high-order interaction information between subgraphs. These SGNs will serve as augmented views, where original/augmented views with consistent origins are regarded as positive samples, and those views with inconsistent origins are regarded as negative samples. In graph representation learning, all views will be fed into the encoder group to obtain node-level representations. The encoders for different order SGN are independent of each other. Through the readout function, these node-level representations are compressed into graph-level representations. A shared projection head projects representations into a more rigid latent space. In the comparative objective function, we first use the classical function to make the original graph representation close to one of its subgraph network representations in the latent space. Further considering the impact of multi-order SGNs, we propose a fused multi-order contrastive objective function that can approach multi-order subgraph network representations simultaneously.
• Figure 3: SGN-Based Graph augmentation process in the transfer learning setting. For a molecule, we transform it into a graph, where nodes contain atom-related information and edges contain bond-related information. We generate its first-order SGN topology based on SGN rules. Further, we expand its node attributes and edge attributes. The node attributes are updated to the combination of atom types of linked node pairs in $G_{ori}$, and the edge attributes are updated to the combination of ' bond-central atom-bond' of the triangular motif in $G_{ori}$.
• Figure 4: Visualization analysis on MUTAG.
• Figure 5: Sensitivity w.r.t. hyperparameter $\boldsymbol{q}$.