Think before Recommendation: Autonomous Reasoning-enhanced Recommender

TL;DR

This work tackles limitations of distillation-based reasoning in recommender systems by introducing RecZero, a pure RL framework that trains a single LLM to jointly analyze users and items, reason about their compatibility, and predict ratings using a structured Think-before-Recommendation prompt. It employs rule-based reward modeling and Group Relative Policy Optimization to guide autonomous reasoning without teacher models, and further proposes RecOne, a hybrid approach that starts from cold-start supervised fine-tuning before RL refinement. Empirical results across Book, Music, and Yelp datasets show RecOne achieving the best performance (up to approximately 12%–30% reductions in RMSE/MAE compared to baselines), while RecZero alone already surpasses prior methods in several metrics and offers significant cost and deployment advantages. The findings demonstrate the viability and practical benefits of RL-driven autonomous reasoning in LLM-based recommender systems, with RecOne providing an even stronger performance ceiling through task-aligned initialization. $R = R_{format} + R_{answer}$ and other reasoning steps are tightly integrated into end-to-end RL, enabling decimal rating predictions without separate post-processing.

Abstract

The core task of recommender systems is to learn user preferences from historical user-item interactions. With the rapid development of large language models (LLMs), recent research has explored leveraging the reasoning capabilities of LLMs to enhance rating prediction tasks. However, existing distillation-based methods suffer from limitations such as the teacher model's insufficient recommendation capability, costly and static supervision, and superficial transfer of reasoning ability. To address these issues, this paper proposes RecZero, a reinforcement learning (RL)-based recommendation paradigm that abandons the traditional multi-model and multi-stage distillation approach. Instead, RecZero trains a single LLM through pure RL to autonomously develop reasoning capabilities for rating prediction. RecZero consists of two key components: (1) "Think-before-Recommendation" prompt construction, which employs a structured reasoning template to guide the model in step-wise analysis of user interests, item features, and user-item compatibility; and (2) rule-based reward modeling, which adopts group relative policy optimization (GRPO) to compute rewards for reasoning trajectories and optimize the LLM. Additionally, the paper explores a hybrid paradigm, RecOne, which combines supervised fine-tuning with RL, initializing the model with cold-start reasoning samples and further optimizing it with RL. Experimental results demonstrate that RecZero and RecOne significantly outperform existing baseline methods on multiple benchmark datasets, validating the superiority of the RL paradigm in achieving autonomous reasoning-enhanced recommender systems.

Figures (5)

• Figure 1: Comparison between RecZero and Conventional Paradigms. Conventional paradigms train multiple student LLMs using diverse query-based datasets with reasoning traces, while RecZero uses pure RL to train a single LLM for the entire workflow. Unlike conventional methods reliant on Teacher models, RecZero leverages Reward Signals to jointly optimize reasoning traces and predictions, improving efficiency and performance.
• Figure 2: The RecZero and RecOne frameworks utilize a unified target to optimize multi-step reasoning within a coupled structure, achieving reasoning-enhanced predictions specifically designed for recommendation scenarios.
• Figure 3: The performance of RecZero and RecOne over training steps.
• Figure 4: \ref{['fig:ab_mae']} and \ref{['fig:ab_rmse']} separately show the MAE and RMSE performance of RecOne and its four ablation variants on four datasets.
• Figure 5: System Prompt