Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Union Subgraph Neural Networks

Jiaxing Xu, Aihu Zhang, Qingtian Bian, Vijay Prakash Dwivedi, Yiping Ke

TL;DR

This work addresses the expressiveness gap of vanilla GNNs by introducing union subgraphs that fully capture the $1$-hop neighborhood of an edge and a shortest-path–based descriptor to encode high-order neighbor connectivities. It proposes UnionSNN, which uses these local encodings in message passing and can be plugged into other MPNNs and Transformer models; theoretically, UnionSNN is strictly more expressive than the $1$-WL test and, in some cases, stronger than $3$-WL. Empirically, UnionSNN achieves state-of-the-art or competitive performance across 18 graph- and node-level benchmarks, with up to 11.09% improvements when injecting its structural coefficients into baseline models, while maintaining competitive efficiency. The work provides a scalable, modular approach to infusing rich local structure into modern graph models.

Abstract

Graph Neural Networks (GNNs) are widely used for graph representation learning in many application domains. The expressiveness of vanilla GNNs is upper-bounded by 1-dimensional Weisfeiler-Leman (1-WL) test as they operate on rooted subtrees through iterative message passing. In this paper, we empower GNNs by injecting neighbor-connectivity information extracted from a new type of substructure. We first investigate different kinds of connectivities existing in a local neighborhood and identify a substructure called union subgraph, which is able to capture the complete picture of the 1-hop neighborhood of an edge. We then design a shortest-path-based substructure descriptor that possesses three nice properties and can effectively encode the high-order connectivities in union subgraphs. By infusing the encoded neighbor connectivities, we propose a novel model, namely Union Subgraph Neural Network (UnionSNN), which is proven to be strictly more powerful than 1-WL in distinguishing non-isomorphic graphs. Additionally, the local encoding from union subgraphs can also be injected into arbitrary message-passing neural networks (MPNNs) and Transformer-based models as a plugin. Extensive experiments on 18 benchmarks of both graph-level and node-level tasks demonstrate that UnionSNN outperforms state-of-the-art baseline models, with competitive computational efficiency. The injection of our local encoding to existing models is able to boost the performance by up to 11.09%. Our code is available at https://github.com/AngusMonroe/UnionSNN.

Union Subgraph Neural Networks

TL;DR

This work addresses the expressiveness gap of vanilla GNNs by introducing union subgraphs that fully capture the -hop neighborhood of an edge and a shortest-path–based descriptor to encode high-order neighbor connectivities. It proposes UnionSNN, which uses these local encodings in message passing and can be plugged into other MPNNs and Transformer models; theoretically, UnionSNN is strictly more expressive than the -WL test and, in some cases, stronger than -WL. Empirically, UnionSNN achieves state-of-the-art or competitive performance across 18 graph- and node-level benchmarks, with up to 11.09% improvements when injecting its structural coefficients into baseline models, while maintaining competitive efficiency. The work provides a scalable, modular approach to infusing rich local structure into modern graph models.

Abstract

Graph Neural Networks (GNNs) are widely used for graph representation learning in many application domains. The expressiveness of vanilla GNNs is upper-bounded by 1-dimensional Weisfeiler-Leman (1-WL) test as they operate on rooted subtrees through iterative message passing. In this paper, we empower GNNs by injecting neighbor-connectivity information extracted from a new type of substructure. We first investigate different kinds of connectivities existing in a local neighborhood and identify a substructure called union subgraph, which is able to capture the complete picture of the 1-hop neighborhood of an edge. We then design a shortest-path-based substructure descriptor that possesses three nice properties and can effectively encode the high-order connectivities in union subgraphs. By infusing the encoded neighbor connectivities, we propose a novel model, namely Union Subgraph Neural Network (UnionSNN), which is proven to be strictly more powerful than 1-WL in distinguishing non-isomorphic graphs. Additionally, the local encoding from union subgraphs can also be injected into arbitrary message-passing neural networks (MPNNs) and Transformer-based models as a plugin. Extensive experiments on 18 benchmarks of both graph-level and node-level tasks demonstrate that UnionSNN outperforms state-of-the-art baseline models, with competitive computational efficiency. The injection of our local encoding to existing models is able to boost the performance by up to 11.09%. Our code is available at https://github.com/AngusMonroe/UnionSNN.
Paper Structure (27 sections, 3 theorems, 11 equations, 7 figures, 12 tables, 1 algorithm)

This paper contains 27 sections, 3 theorems, 11 equations, 7 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Substructure-Enhanced GNNs
  4. Path-Related GNNs
  5. Local Substructures to Empower MPNNs
  6. Message Passing Neural Networks
  7. Local Substructures to Improve MPNNs
  8. Union Isomorphism
  9. UnionSNN
  10. Design of Substructure Descriptor Function
  11. Network Design
  12. Expressive Power of UnionSNN
  13. Experimental Study
  14. Performance on Different Graph Tasks
  15. Ablation Study
  16. ...and 12 more sections

Key Result

Theorem 1

If $S_i \simeq_{union} S_j$, then $S_i \simeq_{overlap} S_j$; but not vice versa.

Figures (7)

  • Figure 1: (a) A pair of non-isomorphic graphs not distinguishable by 1-WL; (b) An example of various local substructures for two adjacent nodes $v$ and $u$.
  • Figure 2: An example that the bijective mapping between the nodes in the subgraphs is not the same as $g$.
  • Figure 3: Three properties that a good substructure descriptor function $f(\cdot)$ should exhibit.
  • Figure 4: Example of two non-isomorphic subgraphs with the same structural coefficient in GraphSNN
  • Figure 5: These two graphs can be distinguished by UnionSNN but not by 3-WL.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2
  • Theorem 3