IceBerg: Debiased Self-Training for Class-Imbalanced Node Classification

TL;DR

It is found that leveraging unlabeled nodes can significantly enhance the performance of GNNs in class-imbalanced and few-shot scenarios, and even small, surgical modifications can lead to substantial performance improvements.

Abstract

Graph Neural Networks (GNNs) have achieved great success in dealing with non-Euclidean graph-structured data and have been widely deployed in many real-world applications. However, their effectiveness is often jeopardized under class-imbalanced training sets. Most existing studies have analyzed class-imbalanced node classification from a supervised learning perspective, but they do not fully utilize the large number of unlabeled nodes in semi-supervised scenarios. We claim that the supervised signal is just the tip of the iceberg and a large number of unlabeled nodes have not yet been effectively utilized. In this work, we propose IceBerg, a debiased self-training framework to address the class-imbalanced and few-shot challenges for GNNs at the same time. Specifically, to figure out the Matthew effect and label distribution shift in self-training, we propose Double Balancing, which can largely improve the performance of existing baselines with just a few lines of code as a simple plug-and-play module. Secondly, to enhance the long-range propagation capability of GNNs, we disentangle the propagation and transformation operations of GNNs. Therefore, the weak supervision signals can propagate more effectively to address the few-shot issue. In summary, we find that leveraging unlabeled nodes can significantly enhance the performance of GNNs in class-imbalanced and few-shot scenarios, and even small, surgical modifications can lead to substantial performance improvements. Systematic experiments on benchmark datasets show that our method can deliver considerable performance gain over existing class-imbalanced node classification baselines. Additionally, due to IceBerg's outstanding ability to leverage unsupervised signals, it also achieves state-of-the-art results in few-shot node classification scenarios. The code of IceBerg is available at: https://github.com/ZhixunLEE/IceBerg.

Figures (6)

• Figure 1: Left: Confidence of pseudo labels in the long-tailed node classification task. The first four classes are majority classes, and the latter three are minority classes. Right: Matthew Effect of standard multi-stage self-training on long-tailed graph datasets.
• Figure 2: Toy example on the two-moon dataset. We utilize a simple two-layer fully connected MLP for classification.
• Figure 3: On the Cora dataset, when the imbalance ratio (IR) is 20, the utilization rate of unlabeled nodes and the accuracy of pseudo labels vary across epochs under different thresholds.
• Figure 4: Visualization of node representations.
• Figure 5: The first row is the class distribution of the unlabeled set, and the second row is that of the labeled set.