Internalizing Multi-Agent Reasoning for Accurate and Efficient LLM-based Recommendation

TL;DR

The paper tackles the challenge of marrying LLM-based semantic reasoning with collaborative signals in recommender systems while preserving real-time inference. It introduces MARS as a multi-agent teacher that verbalizes collaborative signals via a Collaborative Signal Translation mechanism and then distills this agentic reasoning into a single, efficient STAR model through a trajectory-driven pipeline combining Supervised Fine-Tuning and Group Relative Policy Optimization. The resulting STAR surpasses its teacher in performance across diverse real-world datasets and dramatically reduces coordination latency, enabling real-time, reasoning-enabled recommendations. The work contributes a novel internalization framework, a graph-grounded translation of collaborative signals, and a scalable distillation approach that preserves interpretability and efficiency. This approach has practical implications for deploying reasoning-rich recommendations at scale with lower latency and resource demands.

Abstract

Large Language Models (LLMs) are reshaping recommender systems by leveraging extensive world knowledge and semantic reasoning to interpret user intent. However, effectively integrating these capabilities with collaborative signals while avoiding prohibitive inference latency remains a critical bottleneck. To address this, we propose a trajectory-driven internalization framework to develop a Single-agent Trajectory-Aligned Recommender (STAR). Specifically, to internalize complex reasoning capabilities into a single efficient model, we first design a multi-agent teacher system capable of multi-turn tool usage and reflection. This teacher utilizes a Collaborative Signal Translation mechanism to explicitly convert latent behavioral patterns into descriptive natural language evidence to enhance reasoning accuracy. Subsequently, a trajectory-driven distillation pipeline transfers this agentic logic, including planning, tool usage, and self-reflection, into the compact STAR model. Extensive experiments demonstrate that STAR surpasses its teacher by 8.7% to 39.5% while eliminating iterative latency, paving the way for real-time, reasoning-enhanced recommendation.

Figures (8)

• Figure 1: The overall framework of our approach. It consists of two phases: (1) The Multi-Agent Recommender System as a teacher to synthesize reasoning-rich interaction trajectories using external tools; (2) A trajectory-driven internalization pipeline that distills these capabilities into the STAR student model via Supervised Fine-Tuning and reinforcement learning.
• Figure 2: Schematic illustration of the graph-based collaborative tools. (a) The Item-CF tool identifies co-occurring items (e.g., retrieving Dune via shared readers). (b) The User-CF tool aggregates preferences from similar users to summarize shared interests.
• Figure 3: An example of the serialized chain-of-thought format used for training. Different colors represent distinct agent roles. Crucially, the <tool_call> tokens are retained to teach the student model when and how to access collaborative signals.
• Figure 4: Efficiency-Performance Trade-off. The student model (STAR), indicated by the red star) outperforms the teacher (MARS) while achieving significantly lower inference costs on a single GPU. In contrast, the hybrid MARS-Planner only partially alleviates the resource bottleneck, as it still retains dependency on the high-resource backend.
• Figure 5: Performance scaling of STAR on the Goodreads dataset across parameter sizes ranging from 1.7B to 32B. The model exhibits consistent performance gains with increasing size.