Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parallel Latent Reasoning for Sequential Recommendation

Jiakai Tang, Xu Chen, Wen Chen, Jian Wu, Yuning Jiang, Bo Zheng

TL;DR

PLR introduces width-level parallel latent reasoning for sequential recommendation, moving beyond depth-centric improvements by running multiple diverse latent streams guided by learnable trigger tokens. Through global diversity regularization, reasoning-contrastive learning, and adaptive mixture-of-streams aggregation, PLR achieves substantial accuracy gains with modest latency overhead, validated across three real-world datasets. Theoretical analysis links diversity among streams to reduced ensemble error and explains the finite-benefit regime of deeper reasoning due to diversity decay. Empirically, PLR demonstrates strong robustness to sparse data and closes performance gaps with efficient inference, highlighting width-level scaling as a practical catalyst for improvements in latent reasoning–driven recommender systems.

Abstract

Capturing complex user preferences from sparse behavioral sequences remains a fundamental challenge in sequential recommendation. Recent latent reasoning methods have shown promise by extending test-time computation through multi-step reasoning, yet they exclusively rely on depth-level scaling along a single trajectory, suffering from diminishing returns as reasoning depth increases. To address this limitation, we propose \textbf{Parallel Latent Reasoning (PLR)}, a novel framework that pioneers width-level computational scaling by exploring multiple diverse reasoning trajectories simultaneously. PLR constructs parallel reasoning streams through learnable trigger tokens in continuous latent space, preserves diversity across streams via global reasoning regularization, and adaptively synthesizes multi-stream outputs through mixture-of-reasoning-streams aggregation. Extensive experiments on three real-world datasets demonstrate that PLR substantially outperforms state-of-the-art baselines while maintaining real-time inference efficiency. Theoretical analysis further validates the effectiveness of parallel reasoning in improving generalization capability. Our work opens new avenues for enhancing reasoning capacity in sequential recommendation beyond existing depth scaling.

Parallel Latent Reasoning for Sequential Recommendation

TL;DR

PLR introduces width-level parallel latent reasoning for sequential recommendation, moving beyond depth-centric improvements by running multiple diverse latent streams guided by learnable trigger tokens. Through global diversity regularization, reasoning-contrastive learning, and adaptive mixture-of-streams aggregation, PLR achieves substantial accuracy gains with modest latency overhead, validated across three real-world datasets. Theoretical analysis links diversity among streams to reduced ensemble error and explains the finite-benefit regime of deeper reasoning due to diversity decay. Empirically, PLR demonstrates strong robustness to sparse data and closes performance gaps with efficient inference, highlighting width-level scaling as a practical catalyst for improvements in latent reasoning–driven recommender systems.

Abstract

Capturing complex user preferences from sparse behavioral sequences remains a fundamental challenge in sequential recommendation. Recent latent reasoning methods have shown promise by extending test-time computation through multi-step reasoning, yet they exclusively rely on depth-level scaling along a single trajectory, suffering from diminishing returns as reasoning depth increases. To address this limitation, we propose \textbf{Parallel Latent Reasoning (PLR)}, a novel framework that pioneers width-level computational scaling by exploring multiple diverse reasoning trajectories simultaneously. PLR constructs parallel reasoning streams through learnable trigger tokens in continuous latent space, preserves diversity across streams via global reasoning regularization, and adaptively synthesizes multi-stream outputs through mixture-of-reasoning-streams aggregation. Extensive experiments on three real-world datasets demonstrate that PLR substantially outperforms state-of-the-art baselines while maintaining real-time inference efficiency. Theoretical analysis further validates the effectiveness of parallel reasoning in improving generalization capability. Our work opens new avenues for enhancing reasoning capacity in sequential recommendation beyond existing depth scaling.
Paper Structure (41 sections, 5 theorems, 41 equations, 5 figures, 4 tables)

This paper contains 41 sections, 5 theorems, 41 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Problem Formulation
  4. Latent Reasoning-Enhanced Recommendation
  5. Methodology
  6. Parallel Latent Reasoning Backbone
  7. Attention-based Sequence Encoding
  8. Depth-Level Latent Reasoning
  9. Width-Level Parallel Reasoning
  10. Multi-Stream Reasoning Optimization
  11. Global Reasoning Regularization
  12. Reasoning Contrastive Learning
  13. Mixture-of-Reasoning-Streams Aggregation
  14. Training Objective
  15. Theoretical Analysis
  16. ...and 26 more sections

Key Result

Theorem 4.1

Define the ensemble loss $\mathcal{L}_{\text{ens}} = \mathbb{E}_{(\mathcal{S}_u,v)\sim\mathcal{D}}[-\log \bar{p}(v|\mathcal{S}_u)]$ and average individual loss $\bar{\mathcal{L}}_{\text{ind}} = \frac{1}{M}\sum_{m=1}^M \mathbb{E}[-\log \hat{p}_m(v|\mathcal{S}_u)]$. Then: where the specialization benefit$\mathcal{I}(\mathcal{S}_u) \geq 0$ quantifies the gain from diversity, with equality if and onl

Figures (5)

  • Figure 1: Overall architecture of the Parallel Latent Reasoning framework. RPE denotes reasoning position embedding.
  • Figure 2: Parameter sensitivity analysis on the CDs & Vinyl dataset with SASRec backbone.
  • Figure 3: Robustness analysis on the CDs & Vinyl dataset.
  • Figure 4: Performance ceiling analysis on the CDs & Vinyl dataset with different backbones.
  • Figure 5: Attention visualization illustration.

Theorems & Definitions (9)

  • Theorem 4.1: Ensemble Error Decomposition
  • Proposition 4.2: Diversity-Specialization Connection
  • Theorem 4.4: Diversity Decay Under Iteration
  • Corollary 4.5: Refinement-Diversity Trade-off
  • Theorem 4.6: Gating Benefit via Mutual Information
  • proof
  • proof
  • proof
  • proof