Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FSW-GNN: A Bi-Lipschitz WL-Equivalent Graph Neural Network

Yonatan Sverdlov, Yair Davidson, Nadav Dym, Tal Amir

TL;DR

This paper seeks bi-Lipschitz continuity guarantees for MPNNs and demonstrates that, in contrast with standard summation-based MPNNs, which lack bi-Lipschitz properties, the proposed model provides a bi-Lipschitz graph embedding with respect to two standard graph metrics.

Abstract

Many of the most popular graph neural networks fall into the category of message-passing neural networks (MPNNs). Famously, MPNNs' ability to distinguish between graphs is limited to graphs separable by the Weisfeiler-Lemann (WL) graph isomorphism test, and the strongest MPNNs, in terms of separation power, are WL-equivalent. Recently, it was shown that the quality of separation provided by standard WL-equivalent MPNN can be very low, resulting in WL-separable graphs being mapped to very similar, hardly distinguishable features. This paper addresses this issue by seeking bi-Lipschitz continuity guarantees for MPNNs. We demonstrate that, in contrast with standard summation-based MPNNs, which lack bi-Lipschitz properties, our proposed model provides a bi-Lipschitz graph embedding with respect to two standard graph metrics. Empirically, we show that our MPNN is competitive with standard MPNNs for several graph learning tasks and is far more accurate in over-squashing long-range tasks.

FSW-GNN: A Bi-Lipschitz WL-Equivalent Graph Neural Network

TL;DR

This paper seeks bi-Lipschitz continuity guarantees for MPNNs and demonstrates that, in contrast with standard summation-based MPNNs, which lack bi-Lipschitz properties, the proposed model provides a bi-Lipschitz graph embedding with respect to two standard graph metrics.

Abstract

Many of the most popular graph neural networks fall into the category of message-passing neural networks (MPNNs). Famously, MPNNs' ability to distinguish between graphs is limited to graphs separable by the Weisfeiler-Lemann (WL) graph isomorphism test, and the strongest MPNNs, in terms of separation power, are WL-equivalent. Recently, it was shown that the quality of separation provided by standard WL-equivalent MPNN can be very low, resulting in WL-separable graphs being mapped to very similar, hardly distinguishable features. This paper addresses this issue by seeking bi-Lipschitz continuity guarantees for MPNNs. We demonstrate that, in contrast with standard summation-based MPNNs, which lack bi-Lipschitz properties, our proposed model provides a bi-Lipschitz graph embedding with respect to two standard graph metrics. Empirically, we show that our MPNN is competitive with standard MPNNs for several graph learning tasks and is far more accurate in over-squashing long-range tasks.

Paper Structure

This paper contains 31 sections, 12 theorems, 34 equations, 3 figures, 4 tables.

Table of Contents

  1. introduction
  2. Related Works
  3. MPNNs with advanced pooling mechanisms
  4. Bi-Lipschitzness
  5. WL-equivalent metrics
  6. Problem Setting
  7. Vertex-featured graphs
  8. Weisfeiler-Lemann Graph Isomorphism test
  9. Message passing neural networks
  10. WL-metric for graphs
  11. Bi-Lipschitzness
  12. Lipschitzness, Holder, and depth
  13. Main Contributions
  14. The DS metric
  15. Bi-Lipschitz MPNN
  16. ...and 16 more sections

Key Result

Theorem 3.1

[Proof in metric_property] Let ${\rho_{\textup{DS}}}: {{\mathcal{G}_{\leq N}{\left({{\mathbb{R}^d}}\right)}}} \times {{\mathcal{G}_{\leq N}{\left({{\mathbb{R}^d}}\right)}}} \to \mathbb{R}_{\geq 0}$ be as in eqdef_metric_ds. Then ${\rho_{\textup{DS}}}$ is a WL-equivalent metric on ${{\mathcal{G}_{\le

Figures (3)

  • Figure 1: Trees models comparison.
  • Figure 2: Comparison of MPNN models in all three learning tasks for the graph transfer task from bron, with CliquePath, Ring, and CrossRing topologies
  • Figure 3: Dirichlet Energy vs. Number of MPNN layers for various models on the Ring long-range task

Theorems & Definitions (26)

  • Definition : WL graph equivalence
  • Definition : WL-metric
  • Definition : Bi-Lipschitz embedding
  • Definition
  • Theorem 3.1
  • Theorem 3.2: Informal
  • Theorem 3.3
  • Lemma 3.3
  • Definition A.1
  • Definition A.2
  • ...and 16 more