DiffGRM: Diffusion-based Generative Recommendation Model

TL;DR

This work tackles the mismatch between autoregressive SID generation and cross-digit semantics in diffusion-based recommendation by introducing DiffGRM, a masked discrete diffusion model for SID digits. It couples Parallel Semantic Encoding (PSE) with OPQ, On-policy Coherent Noising (OCN) to focus supervision on harder digits, and Confidence-guided Parallel Denoising (CPD) to produce diverse Top-$K$ SID candidates. Empirically, DiffGRM achieves state-of-the-art improvements in $NDCG@10$ across Amazon Sports, Beauty, and Toys datasets, demonstrating strong gains over both generative and discriminative baselines. The approach offers bidirectional cross-digit context and efficient parallel generation, with practical significance for semantic-ID–based recommendation systems and top-$K$ candidate generation.

Abstract

Generative recommendation (GR) is an emerging paradigm that represents each item via a tokenizer as an n-digit semantic ID (SID) and predicts the next item by autoregressively generating its SID conditioned on the user's history. However, two structural properties of SIDs make ARMs ill-suited. First, intra-item consistency: the n digits jointly specify one item, yet the left-to-right causality trains each digit only under its prefix and blocks bidirectional cross-digit evidence, collapsing supervision to a single causal path. Second, inter-digit heterogeneity: digits differ in semantic granularity and predictability, while the uniform next-token objective assigns equal weight to all digits, overtraining easy digits and undertraining hard digits. To address these two issues, we propose DiffGRM, a diffusion-based GR model that replaces the autoregressive decoder with a masked discrete diffusion model (MDM), thereby enabling bidirectional context and any-order parallel generation of SID digits for recommendation. Specifically, we tailor DiffGRM in three aspects: (1) tokenization with Parallel Semantic Encoding (PSE) to decouple digits and balance per-digit information; (2) training with On-policy Coherent Noising (OCN) that prioritizes uncertain digits via coherent masking to concentrate supervision on high-value signals; and (3) inference with Confidence-guided Parallel Denoising (CPD) that fills higher-confidence digits first and generates diverse Top-K candidates. Experiments show consistent gains over strong generative and discriminative recommendation baselines on multiple datasets, improving NDCG@10 by 6.9%-15.5%. Code is available at https://github.com/liuzhao09/DiffGRM.

Figures (6)

• Figure 1: Overview of SID structure and ARM limitations. (a) Intra-item consistency means the $n$ SID digits jointly specify one item. Inter-digit heterogeneity means digits carry different semantic granularity and therefore different prediction difficulty. (b) Under ARMs’ causal left-to-right prediction, once the first digit <233> is fixed, the second digit, the hard digit, is evaluated only under this prefix and is mispredicted as <131>. Because the causal constraint forbids using evidence from later digits, subsequent digits are decoded on this incorrect prefix, propagating the error to <55> and <88>. Ours selects digits by confidence and updates in any order. Given <233>, it fills the higher-confidence third digit <56>, then revisits the second digit conditioned on <233> and <56> to correct it to <134>, and finally completes <90>.
• Figure 2: Overall framework of DiffGRM. (a) Training: an encoder summarizes the user history, and a masked-diffusion decoder predicts missing digits in parallel under masks scheduled by On-policy Coherent Noising (OCN), which concentrates corruption on harder digits. (b) Inference: Confidence-guided Parallel Denoising (CPD) performs a global parallel beam search from a fully masked input, filling higher-confidence digits first to recover complete SIDs.
• Figure 3: Analysis of performance (NDCG@10/Recall@10) w.r.t. effective sample passes (ESP). DiffGRM matches or surpasses the $k$-times coherent-path noising settings under lower ESP, indicating higher sample efficiency. For fairness, on Toys we fix $d_{\text{model}}{=}256$, while the optimal $d_{\text{model}}{=}1024$ runs out of memory for $k{>}3$.
• Figure 4: Analysis of DiffGRM performance (NDCG@10) w.r.t. beam size in CPD.
• Figure 5: Example with a 3-layer codebook ($n{=}3$) and semantic ID ($s_{L_u}^{0}, s_{L_u}^{1}, s_{L_u}^{2}$). Left: all learnable supervision signals for the autoregressive model (ARM). Right: all learnable supervision signals for the masked diffusion model (MDM).