Learning to Model Graph Structural Information on MLPs via Graph Structure Self-Contrasting

TL;DR

This work addresses robustness and generalization in graph learning by removing reliance on explicit message passing. It introduces Graph Structural Self-Contrasting (GSSC), a bi-level, two-network framework that learns a sparsified graph and performs self-contrasting with an MLP backbone, guided by a homophily-based objective. The key contributions include a probabilistic STR-Sparse module that yields differentiable, edge-wise sparsification, a STR-Contrast module that enforces structural smoothness without forward-propagation, and a bi-level optimization scheme that aligns sparsification with downstream supervision. Empirically, GSSC achieves state-of-the-art or competitive results on node and graph classification across diverse datasets, while exhibiting strong robustness to label and structure noise and improved efficiency due to bypassing neighborhood fetching. This approach broadens the applicability of graph learning to settings where GNN-style message passing is undesirable or brittle, offering a practical, scalable alternative for robust graph representations.

Abstract

Recent years have witnessed great success in handling graph-related tasks with Graph Neural Networks (GNNs). However, most existing GNNs are based on message passing to perform feature aggregation and transformation, where the structural information is explicitly involved in the forward propagation by coupling with node features through graph convolution at each layer. As a result, subtle feature noise or structure perturbation may cause severe error propagation, resulting in extremely poor robustness. In this paper, we rethink the roles played by graph structural information in graph data training and identify that message passing is not the only path to modeling structural information. Inspired by this, we propose a simple but effective Graph Structure Self-Contrasting (GSSC) framework that learns graph structural information without message passing. The proposed framework is based purely on Multi-Layer Perceptrons (MLPs), where the structural information is only implicitly incorporated as prior knowledge to guide the computation of supervision signals, substituting the explicit message propagation as in GNNs. Specifically, it first applies structural sparsification to remove potentially uninformative or noisy edges in the neighborhood, and then performs structural self-contrasting in the sparsified neighborhood to learn robust node representations. Finally, structural sparsification and self-contrasting are formulated as a bi-level optimization problem and solved in a unified framework. Extensive experiments have qualitatively and quantitatively demonstrated that the GSSC framework can produce truly encouraging performance with better generalization and robustness than other leading competitors.

Figures (8)

• Figure 1: A comparison between two cases of using graph sparsification for GNNs and MLP-based models. $\mathbf{X}$ denotes the input features, $\mathbf{H}$ denotes the hidden features, and $\mathbf{G}$ is the sparsified subgraph. The forward and backward propagation are marked as black and red lines, respectively.
• Figure 2: Illustration of the proposed structural sparsification network, which consists of three main components: (1) Estimate $\Lambda = \{\lambda_{i,j} | i\in\mathcal{V},j\in\mathcal{N}_i \}$ by sparsification distribution $q_{\phi}\left(\lambda_{i,j} \mid \mathbf{X}, \mathbf{A}\right)$; (2) Obtain the sparsification strategy $\mathbf{M}$ by the weighted fusion; (3) Sample a sparsified subgraph $g^\prime$ from sparsification strategy $\mathbf{M}$ through Gumbel-Softmax sampling.
• Figure 3: Illustration of the proposed GSSC framework, consisting of a structural sparsification network, a multilayer perceptron, two label prediction heads ($f_\omega(\cdot)$ and $g_\gamma(\cdot)$), a structural self-contrasting network, as well as two optimization losses.
• Figure 4: Correlation between accuracy and homophily ratio.
• Figure 5: Visualization of the training process with (Top) and without (Bottom) homophily-oriented objective $\mathcal{H}(\cdot)$.