Generative Diffusion Models for Sequential Recommendations

TL;DR

Modifications to the DiffuRec architecture are introduced by adding offset noise in the diffusion process to improve robustness and incorporating a cross-attention mechanism in the Approximator to better capture relevant user-item interactions.

Abstract

Generative models such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) have shown promise in sequential recommendation tasks. However, they face challenges, including posterior collapse and limited representation capacity. The work by Li et al. (2023) introduces a novel approach that leverages diffusion models to address these challenges by representing item embeddings as distributions rather than fixed vectors. This approach allows for a more adaptive reflection of users' diverse interests and various item aspects. During the diffusion phase, the model converts the target item embedding into a Gaussian distribution by adding noise, facilitating the representation of sequential item distributions and the injection of uncertainty. An Approximator then processes this noisy item representation to reconstruct the target item. In the reverse phase, the model utilizes users' past interactions to reverse the noise and finalize the item prediction through a rounding operation. This research introduces enhancements to the DiffuRec architecture, particularly by adding offset noise in the diffusion process to improve robustness and incorporating a cross-attention mechanism in the Approximator to better capture relevant user-item interactions. These contributions led to the development of a new model, DiffuRecSys, which improves performance. Extensive experiments conducted on three public benchmark datasets demonstrate that these modifications enhance item representation, effectively capture diverse user preferences, and outperform existing baselines in sequential recommendation research.

Figures (3)

• Figure 1: Overview of the diffusion process for sequential recommendation: (A) Injecting noise into the target item after $s$ diffusion steps; (B) Generating new item representation based on user history and the last target item; (C) Reverse phase for target item reconstruction; (D) Rounding phase to map the continuous target representation to discrete item indices.
• Figure 2: The architecture of approximation using cross-attention between the last target item and the user's historical interactions.
• Figure 3: The Reconstructed Target Item Distributions after applying clustering for different users across various datasets.