Graph Similarity Regularized Softmax for Semi-Supervised Node Classification

TL;DR

This work introduces a graph similarity regularized softmax (RGNN) by embedding non-local total variation (NLTV) into the softmax activation for semi-supervised node classification on graphs. The method is instantiated within GCN and GraphSAGE, proving robust generalization across 100 random dataset splits and 20 random parameter initializations, and performing favorably on both assortative and disassortative graphs. The key contributions include the variational framing of softmax with NLTV, an alternating min–max optimization scheme for $\mathscr{A}$ and $\eta$, and an RGNN training loop that jointly updates GNN weights and the NLTV-regularized softmax. Empirically, RGNN improves accuracy over standard baselines and highlights the importance of a well-crafted similarity matrix $\textbf{S}$, with potential for learnable or STD-regularized extensions to further enhance performance in graph-based learning tasks.

Abstract

Graph Neural Networks (GNNs) are powerful deep learning models designed for graph-structured data, demonstrating effectiveness across a wide range of applications.The softmax function is the most commonly used classifier for semi-supervised node classification. However, the softmax function lacks spatial information of the graph structure. In this paper, we propose a graph similarity regularized softmax for GNNs in semi-supervised node classification. By incorporating non-local total variation (TV) regularization into the softmax activation function, we can more effectively capture the spatial information inherent in graphs. The weights in the non-local gradient and divergence operators are determined based on the graph's adjacency matrix. We apply the proposed method into the architecture of GCN and GraphSAGE, testing them on citation and webpage linking datasets, respectively. Numerical experiments demonstrate its good performance in node classification and generalization capabilities. These results indicate that the graph similarity regularized softmax is effective on both assortative and disassortative graphs.

Figures (6)

• Figure 1: GNN vs. RGNN: A structural comparison. (The output shows that node $x_1$ is misclassified by the GNN but correctly classified by the RGNN.)
• Figure 2: Classification accuracy on Citeseer dataset of the RGCN model with varying initial settings of parameters $\tau$,$\lambda$ and $\epsilon$, respectively.
• Figure 3: The combined impact of the scaling factor $\tau$ and regularization parameter $\lambda$ on Citeseer dataset.
• Figure 4: Visualization results of GCN and RGCN on Cora, Citeseer and Pubmed datasets (from left to right).
• Figure 5: An example of constructing an ideal similarity matrix $\textbf{S}$.