Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following

Da Guo, Shijia Wang, Qiang Xiao, Yintao Ren, Weisheng Li, Songpei Xu, Ming Yue, Bin Huang, Guanlin Wu, Chuanjiang Luo

TL;DR

This work introduces Time-Aware Multi-Item Prediction (TAMIP), a novel training paradigm designed to mitigate inherent myopia in generative retrieval, and proposes Condition-Guided Sparse Attention (CGSA), which incorporates business constraints directly into the generative process via sparse attention, without introducing additional inference steps.

Abstract

Generative retrieval has emerged as a promising paradigm in recommender systems, offering superior sequence modeling capabilities over traditional dual-tower architectures. However, in large-scale industrial scenarios, such models often suffer from inherent myopia: due to single-step inference and strict latency constraints, they tend to collapse diverse user intents into locally optimal predictions, failing to capture long-horizon and multi-item consumption patterns. Moreover, real-world retrieval systems must follow explicit retrieval instructions, such as category-level control and policy constraints. Incorporating such instruction-following behavior into generative retrieval remains challenging, as existing conditioning or post-hoc filtering approaches often compromise relevance or efficiency. In this work, we present Climber-Pilot, a unified generative retrieval framework to address both limitations. First, we introduce Time-Aware Multi-Item Prediction (TAMIP), a novel training paradigm designed to mitigate inherent myopia in generative retrieval. By distilling long-horizon, multi-item foresight into model parameters through time-aware masking, TAMIP alleviates locally optimal predictions while preserving efficient single-step inference. Second, to support flexible instruction-following retrieval, we propose Condition-Guided Sparse Attention (CGSA), which incorporates business constraints directly into the generative process via sparse attention, without introducing additional inference steps. Extensive offline experiments and online A/B testing at NetEase Cloud Music, one of the largest music streaming platforms, demonstrate that Climber-Pilot significantly outperforms state-of-the-art baselines, achieving a 4.24\% lift of the core business metric.

Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following

TL;DR

This work introduces Time-Aware Multi-Item Prediction (TAMIP), a novel training paradigm designed to mitigate inherent myopia in generative retrieval, and proposes Condition-Guided Sparse Attention (CGSA), which incorporates business constraints directly into the generative process via sparse attention, without introducing additional inference steps.

Abstract

Generative retrieval has emerged as a promising paradigm in recommender systems, offering superior sequence modeling capabilities over traditional dual-tower architectures. However, in large-scale industrial scenarios, such models often suffer from inherent myopia: due to single-step inference and strict latency constraints, they tend to collapse diverse user intents into locally optimal predictions, failing to capture long-horizon and multi-item consumption patterns. Moreover, real-world retrieval systems must follow explicit retrieval instructions, such as category-level control and policy constraints. Incorporating such instruction-following behavior into generative retrieval remains challenging, as existing conditioning or post-hoc filtering approaches often compromise relevance or efficiency. In this work, we present Climber-Pilot, a unified generative retrieval framework to address both limitations. First, we introduce Time-Aware Multi-Item Prediction (TAMIP), a novel training paradigm designed to mitigate inherent myopia in generative retrieval. By distilling long-horizon, multi-item foresight into model parameters through time-aware masking, TAMIP alleviates locally optimal predictions while preserving efficient single-step inference. Second, to support flexible instruction-following retrieval, we propose Condition-Guided Sparse Attention (CGSA), which incorporates business constraints directly into the generative process via sparse attention, without introducing additional inference steps. Extensive offline experiments and online A/B testing at NetEase Cloud Music, one of the largest music streaming platforms, demonstrate that Climber-Pilot significantly outperforms state-of-the-art baselines, achieving a 4.24\% lift of the core business metric.
Paper Structure (30 sections, 8 equations, 4 figures, 5 tables)

This paper contains 30 sections, 8 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Framework Overview
  5. General Retrieval Capability Pre-Training
  6. Sequence Encoding
  7. Time-Aware Multi-Item Prediction
  8. TAMIP Loss Formulation
  9. Instruction Following via SFT
  10. Construction of Condition-Specific Retrieval Datasets
  11. Condition-Guided Sparse Attention Mechanism
  12. SFT Optimization
  13. Deployment Detail
  14. Pre-Computed Recall Instructions
  15. Batched Multi-Condition Inference
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Overview of the Climber-Pilot framework. (a) Model Architecture. This part details the Time-Aware mask employed in the TAMIP module and illustrates the working mechanism of CGSA. During the SFT stage, the TAMIP branch adopts CGSA to enable instruction-following capability. (b) Pre-Training Pipeline. (c) SFT Pipeline.
  • Figure 2: Illustration of the Climber-Pilot inference process.
  • Figure 3: Effectiveness of TAMIP in alleviating inherent myopia. We compare three training paradigms on the industrial dataset: NIP (red), MIP (orange), and TAMIP (blue). The TAMIP curves demonstrate remarkable long-term stability.
  • Figure 4: A case study demonstrating the instruction-following capability of Climber-Pilot.