Graph Contrastive Learning via Cluster-refined Negative Sampling for Semi-supervised Text Classification

TL;DR

This work proposes an innovative GCL-based method of graph contrastive learning via cluster-refined negative sampling for semi-supervised text classification, namely ClusterText, which combines the pre-trained model Bert with graph neural networks to learn text representations and introduces a clustering refinement strategy, which clusters the learned text representations to obtain pseudo labels.

Abstract

Graph contrastive learning (GCL) has been widely applied to text classification tasks due to its ability to generate self-supervised signals from unlabeled data, thus facilitating model training. However, existing GCL-based text classification methods often suffer from negative sampling bias, where similar nodes are incorrectly paired as negative pairs. This can lead to over-clustering, where instances of the same class are divided into different clusters. To address the over-clustering issue, we propose an innovative GCL-based method of graph contrastive learning via cluster-refined negative sampling for semi-supervised text classification, namely ClusterText. Firstly, we combine the pre-trained model Bert with graph neural networks to learn text representations. Secondly, we introduce a clustering refinement strategy, which clusters the learned text representations to obtain pseudo labels. For each text node, its negative sample set is drawn from different clusters. Additionally, we propose a self-correction mechanism to mitigate the loss of true negative samples caused by clustering inconsistency. By calculating the Euclidean distance between each text node and other nodes within the same cluster, distant nodes are still selected as negative samples. Our proposed ClusterText demonstrates good scalable computing, as it can effectively extract important information from from a large amount of data. Experimental results demonstrate the superiority of ClusterText in text classification tasks.

Figures (3)

• Figure 1: Illustration of over-clustering and ideal clustering. Different colors denote distinct text node categories.In (a), the CVD and Resp nodes are assigned to different clusters in the embedding space.
• Figure 2: The overall process of the proposed ClusterText. The views are generated by applying augmentations to the original graph. Next, the representations learned from the original graph are clustered, and the clustering results are refined through self-correction to obtain pseudo-labels. These pseudo-labels are then used to optimize the negative sample selection process. Finally, the model is trained by combining contrastive loss and cross-entropy loss.
• Figure 3: Red node is the anchor. Its negative sample set consists of dark-colored nodes from both within and outside the cluster. Shaded nodes are selected by the clustering refinement strategy and the self-correction mechanism, and they are likely to share the same true label as the anchor.