Generative Reasoning Recommendation via LLMs

TL;DR

This work addresses the gap in leveraging large language models for generative reasoning recommender systems by introducing GREAM, a unified framework that bridges textual semantics and collaborative signals through Collaborative-Semantic Alignment, Reasoning Curriculum Activation, and Sparse-Regularized Policy Optimization. It enables end-to-end training with verifiable rewards and supports two inference modes: Direct Sequence Generation for efficiency and Sequential Reasoning that yields transparent CoT-based reasoning. The approach is validated on three public datasets, showing consistent gains over strong baselines in both direct retrieval and reasoning-centric metrics, and demonstrates favorable scaling of reasoning performance with compute. Collectively, GREAM offers a practical path toward verifiable, interpretable, and scalable LLM-driven recommendations with robust training stability under sparse feedback.

Abstract

Despite their remarkable reasoning capabilities across diverse domains, large language models (LLMs) face fundamental challenges in natively functioning as generative reasoning recommendation models (GRRMs), where the intrinsic modeling gap between textual semantics and collaborative filtering signals, combined with the sparsity and stochasticity of user feedback, presents significant obstacles. This work explores how to build GRRMs by adapting pre-trained LLMs, which achieves a unified understanding-reasoning-prediction manner for recommendation tasks. We propose GREAM, an end-to-end framework that integrates three components: (i) Collaborative-Semantic Alignment, which fuses heterogeneous textual evidence to construct semantically consistent, discrete item indices and auxiliary alignment tasks that ground linguistic representations in interaction semantics; (ii) Reasoning Curriculum Activation, which builds a synthetic dataset with explicit Chain-of-Thought supervision and a curriculum that progresses through behavioral evidence extraction, latent preference modeling, intent inference, recommendation formulation, and denoised sequence rewriting; and (iii) Sparse-Regularized Group Policy Optimization (SRPO), which stabilizes post-training via Residual-Sensitive Verifiable Reward and Bonus-Calibrated Group Advantage Estimation, enabling end-to-end optimization under verifiable signals despite sparse successes. GREAM natively supports two complementary inference modes: Direct Sequence Recommendation for high-throughput, low-latency deployment, and Sequential Reasoning Recommendation that first emits an interpretable reasoning chain for causal transparency. Experiments on three datasets demonstrate consistent gains over strong baselines, providing a practical path toward verifiable-RL-driven LLM recommenders.

Figures (3)

• Figure 1: Scaling of reasoning performance vs. Reasoning Activation training compute. Pass@Avg increases almost linearly with compute and shows no sign of saturation, indicating that our method has not yet reached its performance ceiling.
• Figure 2: An overall framework of our GREAM. (Left): We construct alignment data and synthesize structured reasoning data via a reverse-reasoning process to perform supervised fine-tuning on LLM. (Right): We employ SRPO for further reinforcement learning training , which enables LLM to learn more effectively from sparse reward signals through our proposed Residual-sensitive Reward and Bonus-Calibrated Advantage.
• Figure 3: Impact of reinforcement learning (SRPO) across recommendation tasks on Instruments (Pass@1).