Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck

TL;DR

An effective Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck (CGRL) for node classification, which can adaptively learn to mask the nodes and edges in the graph to obtain the optimal graph structure representation is proposed.

Abstract

Graph Neural Networks (GNNs) have received extensive research attention due to their powerful information aggregation capabilities. Despite the success of GNNs, most of them suffer from the popularity bias issue in a graph caused by a small number of popular categories. Additionally, real graph datasets always contain incorrect node labels, which hinders GNNs from learning effective node representations. Graph contrastive learning (GCL) has been shown to be effective in solving the above problems for node classification tasks. Most existing GCL methods are implemented by randomly removing edges and nodes to create multiple contrasting views, and then maximizing the mutual information (MI) between these contrasting views to improve the node feature representation. However, maximizing the mutual information between multiple contrasting views may lead the model to learn some redundant information irrelevant to the node classification task. To tackle this issue, we propose an effective Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck (CGRL) for node classification, which can adaptively learn to mask the nodes and edges in the graph to obtain the optimal graph structure representation. Furthermore, we innovatively introduce the information bottleneck theory into GCLs to remove redundant information in multiple contrasting views while retaining as much information as possible about node classification. Moreover, we add noise perturbations to the original views and reconstruct the augmented views by constructing adversarial views to improve the robustness of node feature representation. Extensive experiments on real-world public datasets demonstrate that our method significantly outperforms existing state-of-the-art algorithms.

Figures (7)

• Figure 1: (a) The proportion of papers in different categories on the cora and citeseer datasets. (b) The degree distribution of different nodes on the cora and citeseer datasets. By analyzing the distribution of categories and degrees, we argue that the constructed graph structure has a long-tail problem.
• Figure 2: The overview of the proposed CGRL method. Specifically, we combine node dropout and edge dropout views to obtain a more comprehensive representation, where node dropout view can alleviate the problem of popular bias, and edge dropout can alleviate the problem of misconnection between nodes.
• Figure 3: The impact of different noise rates on experimental results on Cora, Citeseer and PubMed datasets.
• Figure 4: The impact of hyperparameter settings (i.e., $\alpha$ and $\beta$) on experimental results in four data sets (i.e., Cora, Citeer, and PubMed).
• Figure 5: Effect of node-masking view and edge perturbation view on training loss.