Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Topology-Informed Graph Transformer

Yun Young Choi, Sun Woo Park, Minho Lee, Youngho Woo

TL;DR

Topological discrimination remains a challenge for Graph Transformers; TIGT addresses it by injecting topology-informed biases through a learnable topological positional embedding based on universal covers of graphs, a dual-path MPNN, a global attention module, and a graph information layer. The approach is theoretically grounded in covering-space theory and cycle bases, enabling discrimination beyond $3$-WL, with complexity scaling as $O(N^2 + N_E + N_C)$. Empirically, TIGT achieves state-of-the-art or competitive results on synthetic CSL data and large benchmarks such as ZINC-full and PCQM4Mv2, often with fewer parameters. This demonstrates that incorporating topological signals enhances both expressive power and predictive performance for graph-level tasks.

Abstract

Transformers have revolutionized performance in Natural Language Processing and Vision, paving the way for their integration with Graph Neural Networks (GNNs). One key challenge in enhancing graph transformers is strengthening the discriminative power of distinguishing isomorphisms of graphs, which plays a crucial role in boosting their predictive performances. To address this challenge, we introduce 'Topology-Informed Graph Transformer (TIGT)', a novel transformer enhancing both discriminative power in detecting graph isomorphisms and the overall performance of Graph Transformers. TIGT consists of four components: A topological positional embedding layer using non-isomorphic universal covers based on cyclic subgraphs of graphs to ensure unique graph representation: A dual-path message-passing layer to explicitly encode topological characteristics throughout the encoder layers: A global attention mechanism: And a graph information layer to recalibrate channel-wise graph features for better feature representation. TIGT outperforms previous Graph Transformers in classifying synthetic dataset aimed at distinguishing isomorphism classes of graphs. Additionally, mathematical analysis and empirical evaluations highlight our model's competitive edge over state-of-the-art Graph Transformers across various benchmark datasets.

Topology-Informed Graph Transformer

TL;DR

Topological discrimination remains a challenge for Graph Transformers; TIGT addresses it by injecting topology-informed biases through a learnable topological positional embedding based on universal covers of graphs, a dual-path MPNN, a global attention module, and a graph information layer. The approach is theoretically grounded in covering-space theory and cycle bases, enabling discrimination beyond -WL, with complexity scaling as . Empirically, TIGT achieves state-of-the-art or competitive results on synthetic CSL data and large benchmarks such as ZINC-full and PCQM4Mv2, often with fewer parameters. This demonstrates that incorporating topological signals enhances both expressive power and predictive performance for graph-level tasks.

Abstract

Transformers have revolutionized performance in Natural Language Processing and Vision, paving the way for their integration with Graph Neural Networks (GNNs). One key challenge in enhancing graph transformers is strengthening the discriminative power of distinguishing isomorphisms of graphs, which plays a crucial role in boosting their predictive performances. To address this challenge, we introduce 'Topology-Informed Graph Transformer (TIGT)', a novel transformer enhancing both discriminative power in detecting graph isomorphisms and the overall performance of Graph Transformers. TIGT consists of four components: A topological positional embedding layer using non-isomorphic universal covers based on cyclic subgraphs of graphs to ensure unique graph representation: A dual-path message-passing layer to explicitly encode topological characteristics throughout the encoder layers: A global attention mechanism: And a graph information layer to recalibrate channel-wise graph features for better feature representation. TIGT outperforms previous Graph Transformers in classifying synthetic dataset aimed at distinguishing isomorphism classes of graphs. Additionally, mathematical analysis and empirical evaluations highlight our model's competitive edge over state-of-the-art Graph Transformers across various benchmark datasets.
Paper Structure (19 sections, 5 theorems, 17 equations, 1 figure, 7 tables)

This paper contains 19 sections, 5 theorems, 17 equations, 1 figure, 7 tables.

Table of Contents

  1. INTRODUCTION
  2. Preliminary
  3. TOPOLOGY-INFORMED GRAPH TRANSFORMER(TIGT)
  4. Topological positional embedding layer
  5. Encoder layer of TIGT
  6. Mathematical background of TIGT
  7. EXPERIMENTS
  8. CONCLUSION
  9. Mathematical Proofs
  10. Proof of Theorem \ref{['theorem:TIGT']}
  11. Proof of Theorem \ref{['theorem:TIGT_2']}
  12. Proof of Theorem \ref{['theorem:TIGT_3']}
  13. Proof of Theorem \ref{['theorem:TIGT_4']}
  14. Abalation study
  15. Implementation details
  16. ...and 4 more sections

Key Result

Theorem 3.1

Suppose $G$ and $H$ are two graphs with the same number of nodes and edges. Suppose that there exists a cyclic subgraph $C$ that is an element of a cycle basis of $G$ such that satisfies the following two conditions: (1)$C$ does not contain any proper cyclic subgraphs, and (2) any element of a cycle

Figures (1)

  • Figure 1: Overall Architecture of TIGT.

Theorems & Definitions (6)

  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Theorem 3.4
  • Definition 1.1
  • Corollary 1.2