Reasoning While Recommending: Entropy-Guided Latent Reasoning in Generative Re-ranking Models

TL;DR

The Entropy-Guided Latent Reasoning (EGLR) recommendation model is introduced, specifically designed for the high-difficulty nature of list generation by enabling real-time reasoning during generation and being compatible with existing generative re-ranking models to enhance their performance.

Abstract

Reinforcement learning plays a crucial role in generative re-ranking scenarios due to its exploration-exploitation capabilities, but existing generative methods mostly fail to adapt to the dynamic entropy changes in model difficulty during list generation, making it challenging to accurately capture complex preferences. Given that language models have achieved remarkable breakthroughs by integrating reasoning capabilities, we draw on this approach to introduce a latent reasoning mechanism, and experimental validation demonstrates that this mechanism effectively reduces entropy in the model's decision-making process. Based on these findings, we introduce the Entropy-Guided Latent Reasoning (EGLR) recommendation model, which has three core advantages. First, it abandons the "reason first, recommend later" paradigm to achieve "reasoning while recommending", specifically designed for the high-difficulty nature of list generation by enabling real-time reasoning during generation. Second, it implements entropy-guided variable-length reasoning using context-aware reasoning token alongside dynamic temperature adjustment, expanding exploration breadth in reasoning and boosting exploitation precision in recommending to achieve a more precisely adapted exploration-exploitation trade-off. Third, the model adopts a lightweight integration design with no complex independent modules or post-processing, enabling easy adaptation to existing models. Experimental results on two real-world datasets validate the model's effectiveness, and its notable advantage lies in being compatible with existing generative re-ranking models to enhance their performance. Further analyses also demonstrate its practical deployment value and research potential.

Figures (6)

• Figure 1: Entropy change of the generative re-ranking model on different position during auto-regressive generation.
• Figure 2: Entropy change of the generative re-ranking model with latent reasoning during auto-regressive generation.
• Figure 3: Architecture of EGLR: The left branch is the Evaluator, pre-trained on historical data and using a Transformer with dual heads to predict item-wise and list-wise feedback. The right branch is the Generator, consisting of an order-agnostic Encoder and an autoregressive Decoder. It incorporates entropy-guided context-aware reasoning tokens and dynamic temperature adjustment—high temperature for reasoning stages and low temperature for recommending stages. The Evaluator provides reward signals to update the Generator via reinforcement learning.
• Figure 4: Ablation of different model variants (B: backbone generator-evaluator, v1: B + entropy-guided latent reasoning, v2: v1 + context-aware reasoning token, v3: v2 + dynamic temperature).
• Figure 5: Entropy reduction at each position after reasoning.