Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sign is Not a Remedy: Multiset-to-Multiset Message Passing for Learning on Heterophilic Graphs

Langzhang Liang, Sunwoo Kim, Kijung Shin, Zenglin Xu, Shirui Pan, Yuan Qi

TL;DR

This paper tackles the challenge of learning on heterophilic graphs where traditional GNNs underperform due to smoothing effects. It reveals two key flaws of Signed Message Passing (SMP): (i) even with desirable one-hop weights, multi-hop propagation can be undesirable, and (ii) SMP is susceptible to oversmoothing in multi-class settings. To address these issues, it proposes Multiset-to-Multiset GNN (M2M-GNN), which aggregates neighborhood information by mapping neighbor embeddings (a multiset) to a multiset of outputs across multiple chunks, thereby preserving class-wise separation and enhancing discriminability. The authors provide theoretical results showing the limitations of SMP and the mitigating properties of m-2-m aggregation, and validate the approach with extensive experiments on 11 datasets, where M2M-GNN consistently ranks among the top methods and surpasses SMP-based models. The work offers a practical, scalable alternative for heterophily-strong graphs and contributes insights into why partitioned, chunk-based aggregation improves robustness to oversmoothing.

Abstract

Graph Neural Networks (GNNs) have gained significant attention as a powerful modeling and inference method, especially for homophilic graph-structured data. To empower GNNs in heterophilic graphs, where adjacent nodes exhibit dissimilar labels or features, Signed Message Passing (SMP) has been widely adopted. However, there is a lack of theoretical and empirical analysis regarding the limitations of SMP. In this work, we unveil some potential pitfalls of SMP and their remedies. We first identify two limitations of SMP: undesirable representation update for multi-hop neighbors and vulnerability against oversmoothing issues. To overcome these challenges, we propose a novel message passing function called Multiset to Multiset GNN(M2M-GNN). Our theoretical analyses and extensive experiments demonstrate that M2M-GNN effectively alleviates the aforementioned limitations of SMP, yielding superior performance in comparison

Sign is Not a Remedy: Multiset-to-Multiset Message Passing for Learning on Heterophilic Graphs

TL;DR

This paper tackles the challenge of learning on heterophilic graphs where traditional GNNs underperform due to smoothing effects. It reveals two key flaws of Signed Message Passing (SMP): (i) even with desirable one-hop weights, multi-hop propagation can be undesirable, and (ii) SMP is susceptible to oversmoothing in multi-class settings. To address these issues, it proposes Multiset-to-Multiset GNN (M2M-GNN), which aggregates neighborhood information by mapping neighbor embeddings (a multiset) to a multiset of outputs across multiple chunks, thereby preserving class-wise separation and enhancing discriminability. The authors provide theoretical results showing the limitations of SMP and the mitigating properties of m-2-m aggregation, and validate the approach with extensive experiments on 11 datasets, where M2M-GNN consistently ranks among the top methods and surpasses SMP-based models. The work offers a practical, scalable alternative for heterophily-strong graphs and contributes insights into why partitioned, chunk-based aggregation improves robustness to oversmoothing.

Abstract

Graph Neural Networks (GNNs) have gained significant attention as a powerful modeling and inference method, especially for homophilic graph-structured data. To empower GNNs in heterophilic graphs, where adjacent nodes exhibit dissimilar labels or features, Signed Message Passing (SMP) has been widely adopted. However, there is a lack of theoretical and empirical analysis regarding the limitations of SMP. In this work, we unveil some potential pitfalls of SMP and their remedies. We first identify two limitations of SMP: undesirable representation update for multi-hop neighbors and vulnerability against oversmoothing issues. To overcome these challenges, we propose a novel message passing function called Multiset to Multiset GNN(M2M-GNN). Our theoretical analyses and extensive experiments demonstrate that M2M-GNN effectively alleviates the aforementioned limitations of SMP, yielding superior performance in comparison
Paper Structure (41 sections, 11 theorems, 84 equations, 5 figures, 3 tables)

This paper contains 41 sections, 11 theorems, 84 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Limitations of Signed Message Passing
  3. Preliminaries
  4. Limitation 1: Undesirable Embedding Update
  5. Limitation 2: Vulnerability to Oversmoothing
  6. Multiset to Multiset Message Passing
  7. Motivation of M2M-GNN
  8. Proposed Method: M2M-GNN
  9. Experiments
  10. Datasets.
  11. Baseline models.
  12. RQ1: Node Classification Performance Results
  13. RQ2: Attention Weight Analysis
  14. RQ3: Over-smoothing Analysis
  15. RQ4: M2M-GNN Component Analysis
  16. ...and 26 more sections

Key Result

Theorem 2.2

There exists a graph $\mathcal{G}$ with more than two distinct node classes (i.e., $C > 2$) where all propagation matrix $\mathcal{A}^{(1)},\mathcal{A}^{(2)},\cdots , \mathcal{A}^{(K)}$ are desirable (Def. def:good), but the cumulative propagation matrix $\mathcal{T}$ is not desirable.

Figures (5)

  • Figure 1: Simulation results for (a) binary and (b) multi-class classification reveal that, as the number of layers increases, the mean embeddings of different classes converge in the multi-class case but not in the binary case. That is, SMP suffers from oversmoothing in the multi-class case, as also supported by (c) the drop of $z^{(k)}$, a score positively associated with the accuracy of the Bayes optimal classifier.
  • Figure 2: An illustration of the convolution layer of M2M-GNN , where the ego (central) node is $v_{1}$. For a given graph topology and node features (left), we first project node features from $3$-dimension to $1$-dimension (middle). Then, we construct a message vector for node $v_{1}$ by concatenating $3$ chunks, each of which consists of a subset of neighbors' projected features using soft labels
  • Figure 3: Visualization results of $\mathbf{\mathcal{S}}$ for two homophilic (left two) and two heterophilic (right two) graphs. (Off-)Diagonal entries indicate (in)correct prediction.
  • Figure 4: Performance comparison of M2M-GNN against SMP-based GNNs under various model depths. The X-axis has the number of layers, and the Y-axis has node classification accuracy.
  • Figure 5: Ablation study. Each tuple on the X-axis indicates a pair of parameters: (number of chunks $\mathcal{C}$, strength of regularization $\lambda$). For all the metrics, higher values indicate better performance.

Theorems & Definitions (25)

  • Definition 2.1: Desirable matrix
  • Theorem 2.2: Undesirability of SMP in multi-class cases
  • Definition 2.3: CSBM for $\mathcal{A}$ and $\mathbf{X}$
  • Theorem 2.4: Oversmoothing problem of SMP
  • Definition 3.1
  • Definition 3.2: m-2-m schemes
  • Definition 3.3: Desirable m-2-m schemes
  • Definition 3.4: Desirable m-2-m message passing
  • Lemma 3.5: Maintenance of desirable property
  • Lemma 3.6: m-2-m is always no worse than m-2-e (informal)
  • ...and 15 more