Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Reflect and Correct: Towards Better Decoding Trajectories for Large-Scale Generative Recommendation

Haibo Xing, Hao Deng, Lingyu Mu, Jinxin Hu, Yu Zhang, Xiaoyi Zeng, Jing Zhang

TL;DR

This work proposes GRC, the first structured reflection-correction framework for GR that extends standard decoding into a Generation-Reflection-Correction (GRC) process, and introduces a supervised reflection-correction template that decomposes the decoding process into initial draft generation, multi-granular reflection, and reflection-guided correction, thereby enabling structured reflection and correction in the semantic token space.

Abstract

Generative Recommendation (GR) has become a promising paradigm for large-scale recommendation systems. However, existing GR models typically perform single-pass decoding without explicit refinement, causing early deviations to accumulate and ultimately degrade recommendation quality. To tackle this problem, we propose GRC, which is, to our knowledge, the first structured reflection-correction framework for GR that extends standard decoding into a Generation-Reflection-Correction (GRC) process. Concretely, GRC introduces a supervised reflection-correction template that decomposes the decoding process into initial draft generation, multi-granular reflection, and reflection-guided correction, thereby enabling structured reflection and correction in the semantic token space. To further explore the enlarged refinement space introduced by the GRC process, we optimize the entire GRC trajectory with GRPO-based reinforcement learning, under a carefully designed reward function with token-level and trajectory-level signals. For efficient online serving, we propose an Entropy-Guided Reflection Scheduling (EGRS) strategy that dynamically allocates more correction budget to high-uncertainty decoding trajectories during beam search. Extensive experiments on real-world datasets show that GRC consistently outperforms six state-of-the-art baselines by up to 15.74%, and online A/B tests demonstrate its substantial practical value in large-scale industrial recommendation, delivering a 1.79% lift in advertising revenue with only modest latency overhead.

Learning to Reflect and Correct: Towards Better Decoding Trajectories for Large-Scale Generative Recommendation

TL;DR

This work proposes GRC, the first structured reflection-correction framework for GR that extends standard decoding into a Generation-Reflection-Correction (GRC) process, and introduces a supervised reflection-correction template that decomposes the decoding process into initial draft generation, multi-granular reflection, and reflection-guided correction, thereby enabling structured reflection and correction in the semantic token space.

Abstract

Generative Recommendation (GR) has become a promising paradigm for large-scale recommendation systems. However, existing GR models typically perform single-pass decoding without explicit refinement, causing early deviations to accumulate and ultimately degrade recommendation quality. To tackle this problem, we propose GRC, which is, to our knowledge, the first structured reflection-correction framework for GR that extends standard decoding into a Generation-Reflection-Correction (GRC) process. Concretely, GRC introduces a supervised reflection-correction template that decomposes the decoding process into initial draft generation, multi-granular reflection, and reflection-guided correction, thereby enabling structured reflection and correction in the semantic token space. To further explore the enlarged refinement space introduced by the GRC process, we optimize the entire GRC trajectory with GRPO-based reinforcement learning, under a carefully designed reward function with token-level and trajectory-level signals. For efficient online serving, we propose an Entropy-Guided Reflection Scheduling (EGRS) strategy that dynamically allocates more correction budget to high-uncertainty decoding trajectories during beam search. Extensive experiments on real-world datasets show that GRC consistently outperforms six state-of-the-art baselines by up to 15.74%, and online A/B tests demonstrate its substantial practical value in large-scale industrial recommendation, delivering a 1.79% lift in advertising revenue with only modest latency overhead.
Paper Structure (28 sections, 25 equations, 3 figures, 3 tables)

This paper contains 28 sections, 25 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generative Recommendation
  4. Self-Reflection in Large Language Models
  5. Preliminaries
  6. Problem Definition
  7. Pre-trained Generative Retrieval Backbone
  8. Methodology
  9. Supervised Fine-tuning with a Structured Reflection-Correction Template
  10. Multi-granular reflection labels
  11. (1) Token-level Error Localization Label.
  12. (2) Semantic-level Consistency Labels.
  13. Structured Reflection-Correction Template
  14. Initial prediction segment.
  15. Reflection segment.
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Illustration of Generative Recommendation. (a) Classic GR, where errors accumulate along one-pass autoregressive generation. (b) Our GRC, which augments generation with structured reflection and correction.
  • Figure 2: Overview of the proposed GRC framework, which extends standard generative retrieval into a structured Generation–Reflection–Correction process with (a). Supervised reflection–correction template, (b). GRPO-based optimization of self–correction, and (c). Entropy-Guided Reflection Scheduling integrated with beam search for practical deployment.
  • Figure 3: Sensitivity Analysis