GRank: Towards Target-Aware and Streamlined Industrial Retrieval with a Generate-Rank Framework

TL;DR

GRank introduces an index-free Generat e→Rank retrieval paradigm that unifies target-aware candidate generation with a lightweight, end-to-end trainable Ranker. By training a Target-Aware Enhanced Generator alongside a cross-attention Ranker and employing a decomposed causal self-attention mechanism, GRank achieves substantial gains in Recall@K while meeting industrial latency constraints. The two-stage online pipeline leverages GPU-accelerated MIPS to prune a large corpus and a fast, candidate-specific scorer to refine results, avoiding the maintenance burdens of structured indices. Extensive offline and online experiments across public and industrial corpora, plus production deployment on billions-scale traffic, demonstrate consistent improvements in retrieval quality, throughput, and core engagement metrics. This work offers a scalable, practical path to bridging accuracy and efficiency in large-scale industrial retrieval systems.

Abstract

Industrial-scale recommender systems rely on a cascade pipeline in which the retrieval stage must return a high-recall candidate set from billions of items under tight latency. Existing solutions ei- ther (i) suffer from limited expressiveness in capturing fine-grained user-item interactions, as seen in decoupled dual-tower architectures that rely on separate encoders, or generative models that lack precise target-aware matching capabilities, or (ii) build structured indices (tree, graph, quantization) whose item-centric topologies struggle to incorporate dynamic user preferences and incur prohibitive construction and maintenance costs. We present GRank, a novel structured-index-free retrieval paradigm that seamlessly unifies target-aware learning with user-centric retrieval. Our key innovations include: (1) A target-aware Generator trained to perform personalized candidate generation via GPU-accelerated MIPS, eliminating semantic drift and maintenance costs of structured indexing; (2) A lightweight but powerful Ranker that performs fine-grained, candidate-specific inference on small subsets; (3) An end-to-end multi-task learning framework that ensures semantic consistency between generation and ranking objectives. Extensive experiments on two public benchmarks and a billion-item production corpus demonstrate that GRank improves Recall@500 by over 30% and 1.7$\times$ the P99 QPS of state-of-the-art tree- and graph-based retrievers. GRank has been fully deployed in production in our recommendation platform since Q2 2025, serving 400 million active users with 99.95% service availability. Online A/B tests confirm significant improvements in core engagement metrics, with Total App Usage Time increasing by 0.160% in the main app and 0.165% in the Lite version.

Figures (3)

• Figure 1: GRank Framework Architecture. The framework comprises two tightly-coupled modules optimized through three complementary losses: (1) Target-Aware Enhanced Generator integrating Generator (optimized by $\mathcal{L}_{\text{NTP}}$) for short-term preference modeling and Auxiliary (optimized by $\mathcal{L}_{\text{SA-info}}$) for target injection; (2) Ranker (optimized by $\mathcal{L}_{\text{CA-info}}$) employing efficient cross-attention for fast inference. The Auxiliary module injects target embedding $\mathbf{C}^*$ and in-batch negatives $\mathbf{C_j}$ during training only. The Generator utilizes our novel decomposed causal self-attention (detailed in bottom-right) to efficiently process user sequences alongside candidate items while maintaining mathematical equivalence. All components are jointly optimized through multi-task learning.
• Figure 2: GRank Serving Architecture. Stage-1 (Generator): Leverages user latent interests to generate a personalized candidate subset. Starge-2 (Ranker):Employs a target-aware cross-attention scorer for fine-grained relevance estimation.
• Figure 3: Hyper-parameter analysis on the efficiency-performance trade-off. (a) Sequence length: CA Recall@500 increases monotonically with $L$, peaking when $L\!=\!1\,000$, significantly outperforming the SA module while maintaining lower latency. (b) Candidate size: diminishing returns begin at $k_1\!=\!2\,000$; recall improves by only $0.1$ pp at $k_1\!=\!5\,000$ while QPS drops $18\%$, making $2\,000$ the pragmatic optimum. (c) Top embedding dimension: recall saturates at $d_{\text{top}}\!=\!128$ and falls at $256$ owing to memory pressure; $d_{\text{top}}\!=\!64$ offers a high-ROI fallback, trading $3\%$ recall for $56\%$ QPS gain.