Position-aware Graph Transformer for Recommendation

TL;DR

This work tackles the challenge of capturing long-range collaborative filtering signals in graph-based recommendations by introducing PGTR, a position-aware graph transformer. PGTR couples a GCN-based backbone with a Transformer module (via Nodeformer Convolution) and four tailored positional encodings (Spectral, Degree, PageRank, and Type) to fuse local neighborhood information with global relationships. The framework is model-agnostic, can be plugged into backbones like LightGCN or NGCF, and is trained with the sampled softmax loss, achieving improved accuracy and robustness to sparsity and noise across four real-world datasets. Experiments also show that each positional encoding contributes to performance, and a controlled mixture of local and global information is essential for best results. Overall, PGTR demonstrates that explicit structural positional information combined with global modeling yields practical gains in recommendation systems, especially under data sparsity and noise conditions.

Abstract

Collaborative recommendation fundamentally involves learning high-quality user and item representations from interaction data. Recently, graph convolution networks (GCNs) have advanced the field by utilizing high-order connectivity patterns in interaction graphs, as evidenced by state-of-the-art methods like PinSage and LightGCN. However, one key limitation has not been well addressed in existing solutions: capturing long-range collaborative filtering signals, which are crucial for modeling user preference. In this work, we propose a new graph transformer (GT) framework -- \textit{Position-aware Graph Transformer for Recommendation} (PGTR), which combines the global modeling capability of Transformer blocks with the local neighborhood feature extraction of GCNs. The key insight is to explicitly incorporate node position and structure information from the user-item interaction graph into GT architecture via several purpose-designed positional encodings. The long-range collaborative signals from the Transformer block are then combined linearly with the local neighborhood features from the GCN backbone to enhance node embeddings for final recommendations. Empirical studies demonstrate the effectiveness of the proposed PGTR method when implemented on various GCN-based backbones across four real-world datasets, and the robustness against interaction sparsity as well as noise.

Figures (8)

• Figure 1: An example of long-range collaborative signals in movie recommendation, where the target user may be interested in the movie Once Upon a Time in America seven hops away in the user-item graph. The dashed lines between nodes are the user-item interactions.
• Figure 2: Overall framework of our PGTR model.
• Figure 3: The procedure of Degree Encoding for users and items.
• Figure 4: The procedure of PageRank Encoding for users and items.
• Figure 5: Performance comparison on Douban-book and LastFM datasets under different proportions of the training set.