GQWformer: A Quantum-based Transformer for Graph Representation Learning

TL;DR

GQWformer introduces a Graph Quantum Walk Transformer that integrates discrete-time quantum walks to embed graph topology into attention biases. By coupling a Graph Quantum Walk Self-attention (GQW-Attn) with a Graph Quantum Walk Recurrent module (GQW-Recu), the model captures both global relations and local structural nuances for graph classification. Empirical results on five TU benchmarks show competitive or superior performance against strong baselines, including recent quantum-inspired methods, with ablations confirming the value of the QW-based encodings and the bidirectional recurrence. This work demonstrates the potential of combining quantum-inspired structural encodings with Transformer architectures to enhance graph representation learning and suggests avenues for future quantum-enhanced GNNs.

Abstract

Graph Transformers (GTs) have demonstrated significant advantages in graph representation learning through their global attention mechanisms. However, the self-attention mechanism in GTs tends to neglect the inductive biases inherent in graph structures, making it chanllenging to effectively capture essential structural information. To address this issue, we propose a novel approach that integrate graph inductive bias into self-attention mechanisms by leveraging quantum technology for structural encoding. In this paper, we introduce the Graph Quantum Walk Transformer (GQWformer), a groundbreaking GNN framework that utilizes quantum walks on attributed graphs to generate node quantum states. These quantum states encapsulate rich structural attributes and serve as inductive biases for the transformer, thereby enabling the generation of more meaningful attention scores. By subsequently incorporating a recurrent neural network, our design amplifies the model's ability to focus on both local and global information. We conducted comprehensive experiments across five publicly available datasets to evaluate the effectiveness of our model. These results clearly indicate that GQWformer outperforms existing state-of-the-art graph classification algorithms. These findings highlight the significant potential of integrating quantum computing methodologies with traditional GNNs to advance the field of graph representation learning, providing a promising direction for future research and applications.

Figures (3)

• Figure 1: The comparison of encoding strategies. In graphs $G_1$ and $G_2$, the positional encodings between nodes $u$ and $v$ based on shortest paths are identical, yet their structural encodings differ due to distinct structures. Conversely, in $G_2$ and $G_3$, their structural encodings are identical when considering only the pure structural information. However, if the node attributes or edge weights differ, their attribute-aware structural encodings should vary.
• Figure 2: GQWformer. The model is composed of several GQWformer blocks. Each block consists of a GQW Generator, a Feed Forward Module (FFN), a GQW-Attn, a GQW-Recu, and a second FFN. With QW encoding, the structural information is effectively injected into the GQWformer.
• Figure 3: The architecture of the Graph Quantum Walk Recurrent Module