Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Directional Diffusion Graph Transformer for Recommendation

Zixuan Yi, Xi Wang, Iadh Ounis

TL;DR

Implicit user feedback in recommender systems is noisy, and isotropic diffusion in graph-based methods can blur item distinctions. We propose DiffGT, a diffusion-based graph transformer that uses directional Gaussian noise in the forward phase and a conditioned denoising reverse phase to recover user preferences. The approach includes a graph-encoder with side information, a linear transformer for efficient reverse diffusion, and a conditioning signal derived from user interactions; it supports continuous diffusion with sampling for efficiency and applies to knowledge-graph and sequential recommendation tasks. Experiments on Movielens-1M, Foursquare, and Yelp2018 show DiffGT outperforms ten baselines, with ablation confirming the value of directional noise, conditioning, and the transformer-based denoiser, and demonstrates scalable diffusion with generalization to KG and sequential settings.

Abstract

In real-world recommender systems, implicitly collected user feedback, while abundant, often includes noisy false-positive and false-negative interactions. The possible misinterpretations of the user-item interactions pose a significant challenge for traditional graph neural recommenders. These approaches aggregate the users' or items' neighbours based on implicit user-item interactions in order to accurately capture the users' profiles. To account for and model possible noise in the users' interactions in graph neural recommenders, we propose a novel Diffusion Graph Transformer (DiffGT) model for top-k recommendation. Our DiffGT model employs a diffusion process, which includes a forward phase for gradually introducing noise to implicit interactions, followed by a reverse process to iteratively refine the representations of the users' hidden preferences (i.e., a denoising process). In our proposed approach, given the inherent anisotropic structure observed in the user-item interaction graph, we specifically use anisotropic and directional Gaussian noises in the forward diffusion process. Our approach differs from the sole use of isotropic Gaussian noises in existing diffusion models. In the reverse diffusion process, to reverse the effect of noise added earlier and recover the true users' preferences, we integrate a graph transformer architecture with a linear attention module to denoise the noisy user/item embeddings in an effective and efficient manner. In addition, such a reverse diffusion process is further guided by personalised information (e.g., interacted items) to enable the accurate estimation of the users' preferences on items. Our extensive experiments conclusively demonstrate the superiority of our proposed graph diffusion model over ten existing state-of-the-art approaches across three benchmark datasets.

A Directional Diffusion Graph Transformer for Recommendation

TL;DR

Implicit user feedback in recommender systems is noisy, and isotropic diffusion in graph-based methods can blur item distinctions. We propose DiffGT, a diffusion-based graph transformer that uses directional Gaussian noise in the forward phase and a conditioned denoising reverse phase to recover user preferences. The approach includes a graph-encoder with side information, a linear transformer for efficient reverse diffusion, and a conditioning signal derived from user interactions; it supports continuous diffusion with sampling for efficiency and applies to knowledge-graph and sequential recommendation tasks. Experiments on Movielens-1M, Foursquare, and Yelp2018 show DiffGT outperforms ten baselines, with ablation confirming the value of directional noise, conditioning, and the transformer-based denoiser, and demonstrates scalable diffusion with generalization to KG and sequential settings.

Abstract

In real-world recommender systems, implicitly collected user feedback, while abundant, often includes noisy false-positive and false-negative interactions. The possible misinterpretations of the user-item interactions pose a significant challenge for traditional graph neural recommenders. These approaches aggregate the users' or items' neighbours based on implicit user-item interactions in order to accurately capture the users' profiles. To account for and model possible noise in the users' interactions in graph neural recommenders, we propose a novel Diffusion Graph Transformer (DiffGT) model for top-k recommendation. Our DiffGT model employs a diffusion process, which includes a forward phase for gradually introducing noise to implicit interactions, followed by a reverse process to iteratively refine the representations of the users' hidden preferences (i.e., a denoising process). In our proposed approach, given the inherent anisotropic structure observed in the user-item interaction graph, we specifically use anisotropic and directional Gaussian noises in the forward diffusion process. Our approach differs from the sole use of isotropic Gaussian noises in existing diffusion models. In the reverse diffusion process, to reverse the effect of noise added earlier and recover the true users' preferences, we integrate a graph transformer architecture with a linear attention module to denoise the noisy user/item embeddings in an effective and efficient manner. In addition, such a reverse diffusion process is further guided by personalised information (e.g., interacted items) to enable the accurate estimation of the users' preferences on items. Our extensive experiments conclusively demonstrate the superiority of our proposed graph diffusion model over ten existing state-of-the-art approaches across three benchmark datasets.
Paper Structure (10 sections, 12 equations, 5 figures, 4 tables)

This paper contains 10 sections, 12 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. A Directional Diffusion Model
  4. Preliminaries
  5. Top-$k$ Recommendation Task
  6. Directional Gaussian Noise
  7. Diffusion Graph Transformer (DiffGT)
  8. Experiments
  9. RESULTS AND ANALYSIS
  10. Conclusions

Figures (5)

  • Figure 1: 2D visualisation of the data using SVD decomposition. (a) Visualisation of the item representations in the MovieLens-1M dataset, with different colours indicating the genres of movies. (b) Visualisation of all tags of venues in the Foursquare dataset.
  • Figure 2: An illustration of our DiffGT architecture.
  • Figure 3: The Signal-to-Noise Ratio (SNR) curves along different diffusion steps on MovieLens-1M and Foursquare.
  • Figure 4: Recommendation performance of different KG and sequential recommendation models with or without our directional noise and the used linear transformer. A red colour on the bar indicates the significant difference between the tested models and their corresponding directional diffusion-enhanced approaches based on the paired t-test with $p<0.05$.
  • Figure :