DiffuReason: Bridging Latent Reasoning and Generative Refinement for Sequential Recommendation

TL;DR

DiffuReason introduces a unified Think-then-Diffuse framework for sequential recommendation that combines explicit latent reasoning with a diffusion-based refinement to model uncertainty in user intent. By generating Thinking Tokens, applying probabilistic denoising, and aligning the process end-to-end via Group Relative Policy Optimization, it jointly optimizes backbone representations, reasoning, and ranking objectives. The approach yields consistent improvements across diverse backbones and domains, with notable gains on challenging architectures like HSTU and strong online validation in a large-scale platform. This work demonstrates that end-to-end diffusion-driven refinement can robustly denoise latent reasoning and translate into tangible gains in real-world recommendations, suggesting practical benefits for production systems and avenues for future multi-modal extensions.

Abstract

Latent reasoning has emerged as a promising paradigm for sequential recommendation, enabling models to capture complex user intent through multi-step deliberation. Yet existing approaches often rely on deterministic latent chains that accumulate noise and overlook the uncertainty inherent in user intent, and they are typically trained in staged pipelines that hinder joint optimization and exploration. To address these challenges, we propose DiffuReason, a unified "Think-then-Diffuse" framework for sequential recommendation. It integrates multi-step Thinking Tokens for latent reasoning, diffusion-based refinement for denoising intermediate representations, and end-to-end Group Relative Policy Optimization (GRPO) alignment to optimize for ranking performance. In the Think stage, the model generates Thinking Tokens that reason over user history to form an initial intent hypothesis. In the Diffuse stage, rather than treating this hypothesis as the final output, we refine it through a diffusion process that models user intent as a probabilistic distribution, providing iterative denoising against reasoning noise. Finally, GRPO-based reinforcement learning enables the reasoning and refinement modules to co-evolve throughout training, without the constraints of staged optimization. Extensive experiments on four benchmarks demonstrate that DiffuReason consistently improves diverse backbone architectures. Online A/B tests on a large-scale industrial platform further validate its practical effectiveness.

Figures (7)

• Figure 1: The overall architecture of the proposed method. The Latent Reasoning module generates explicit Thinking Tokens, which are aggregated via Pooling to form the Condition $c$. (Middle) The Refining Module, driven by a Denoising Module, iteratively recovers the latent state $x^0$ to produce a deterministic Anchor $\mu$ and stochastic samples. The Decoder maps these Refining Tokens to predictions, which are optimized jointly by the recommend loss (Rec Loss) and the alignment reward.
• Figure 2: Efficiency analysis of DiffuReason. (a) Performance (Recall/NDCG) and normalized inference latency under different reasoning steps $R$. (b) Performance (Recall/NDCG) and normalized per-batch training time under different RL sample sizes $G$. Bars denote normalized cost relative to the HSTU backbone.
• Figure 3: Sensitivity to loss weights $\alpha$ (MSE) and $\beta$ (RL) on Beauty and Sports. We vary one weight while fixing the other to 1.0, and report Recall@10 and NDCG@10.
• Figure 4: Visualization of the Think-then-Diffuse trajectory. Top: rank of the ground-truth item across Reasoning steps and the Diffusion stage (lower is better). Bottom: cosine similarity between latent representations at different steps.
• Figure 5: Performance Improvement across user/item subgroups. Users are grouped by sequence length (UG-0 to UG-3), and items are grouped by interaction frequency (IG-0 to IG-3). Bars report metric values at Step 1--3 and Diffusion; the line shows relative improvement over Step 1 within each group.