Masked Diffusion Generative Recommendation

TL;DR

MDGR tackles key drawbacks of autoregressive generative recommendations by reframing SID generation as a masked diffusion process. It introduces OPQ-based parallel codebooks for multi-token SIDs, a training curriculum that gradually increases masking along temporal and sample dimensions, and a warm-up–based two-stage inference with confidence-guided token selection and beam search. Empirically, MDGR achieves state-of-the-art results on public and industrial datasets, with notable improvements up to 10.78% over baselines and measurable online revenue gains. The work demonstrates that diffusion-style masking can capture global dependencies, accommodate heterogeneous user interests, and enable efficient real-time recommendations, pointing to a new direction for diffusion-enabled GR.

Abstract

Generative recommendation (GR) typically first quantizes continuous item embeddings into multi-level semantic IDs (SIDs), and then generates the next item via autoregressive decoding. Although existing methods are already competitive in terms of recommendation performance, directly inheriting the autoregressive decoding paradigm from language models still suffers from three key limitations: (1) autoregressive decoding struggles to jointly capture global dependencies among the multi-dimensional features associated with different positions of SID; (2) using a unified, fixed decoding path for the same item implicitly assumes that all users attend to item attributes in the same order; (3) autoregressive decoding is inefficient at inference time and struggles to meet real-time requirements. To tackle these challenges, we propose MDGR, a Masked Diffusion Generative Recommendation framework that reshapes the GR pipeline from three perspectives: codebook, training, and inference. (1) We adopt a parallel codebook to provide a structural foundation for diffusion-based GR. (2) During training, we adaptively construct masking supervision signals along both the temporal and sample dimensions. (3) During inference, we develop a warm-up-based two-stage parallel decoding strategy for efficient generation of SIDs. Extensive experiments on multiple public and industrial-scale datasets show that MDGR outperforms ten state-of-the-art baselines by up to 10.78%. Furthermore, by deploying MDGR on a large-scale online advertising platform, we achieve a 1.20% increase in revenue, demonstrating its practical value. The code will be released upon acceptance.

Figures (4)

• Figure 1: (a) The codebook quantizes the multimodal information of an item into a sequence of semantic tokens, i.e., SIDs. (b) Autoregressive GR generates SIDs in a fixed left‑to‑right order. (c) Parallel GR generates all tokens in a single step. (d) Our masked diffusion GR denoises multiple positions in parallel, flexibly filling tokens without a fixed order.
• Figure 2: The overview of MDGR: a masked diffusion generative recommendation. (1) Codebook: we adopt an OPQ-based parallel codebook to obtain SIDs for items. (2) Training: we use an encoder–decoder architecture. Based on the current training stage, we first determine the number of masks via global curriculum noise scheduling, and then derive the masked positions for each sample using history-aware mask allocation. (3) Inference: we employ a warm-up–based two-stage parallel decoding strategy, combined with beam search, to jointly generate Top‑B candidate items across multiple codebooks.
• Figure 3: (a) Effect of the $\gamma$ on the global difficulty schedule. (left). (b) Empirical distribution of masked‑token counts $k$ over training steps when $\gamma=2$ (right).
• Figure 4: (a) Effect of the curriculum exponent $\gamma$ on Recall (left). (b) Effect of $\gamma$ on NDCG (right).