Not All Candidates are Created Equal: A Heterogeneity-Aware Approach to Pre-ranking in Recommender Systems

TL;DR

Heterogeneity-Aware Adaptive Pre-ranking (HAP) is presented, a unified framework that mitigates gradient conflicts through conflict-sensitive sampling coupled with tailored loss design, while adaptively allocating computational budgets across candidates.

Abstract

Most large-scale recommender systems follow a multi-stage cascade of retrieval, pre-ranking, ranking, and re-ranking. A key challenge at the pre-ranking stage arises from the heterogeneity of training instances sampled from coarse-grained retrieval results, fine-grained ranking signals, and exposure feedback. Our analysis reveals that prevailing pre-ranking methods, which indiscriminately mix heterogeneous samples, suffer from gradient conflicts: hard samples dominate training while easy ones remain underutilized, leading to suboptimal performance. We further show that the common practice of uniformly scaling model complexity across all samples is inefficient, as it overspends computation on easy cases and slows training without proportional gains. To address these limitations, this paper presents Heterogeneity-Aware Adaptive Pre-ranking (HAP), a unified framework that mitigates gradient conflicts through conflict-sensitive sampling coupled with tailored loss design, while adaptively allocating computational budgets across candidates. Specifically, HAP disentangles easy and hard samples, directing each subset along dedicated optimization paths. Building on this separation, it first applies lightweight models to all candidates for efficient coverage, and further engages stronger models on the hard ones, maintaining accuracy while reducing cost. This approach not only improves pre-ranking effectiveness but also provides a practical perspective on scaling strategies in industrial recommender systems. HAP has been deployed in the Toutiao production system for 9 months, yielding up to 0.4% improvement in user app usage duration and 0.05% in active days, without additional computational cost. We also release a large-scale industrial hybrid-sample dataset to enable the systematic study of source-driven candidate heterogeneity in pre-ranking.

Figures (7)

• Figure 1: Gradient conflicts across negatives of varying difficulty, illustrating the inherent heterogeneity of candidates.
• Figure 2: Architecture of HAP’s training sample collection and model training. Samples are drawn from complete requests, including exposed positives and negatives, and negatives from retrieval, pre-ranking , and ranking stages. HAP adopts the DAMR architecture: the lightweight model learns from all sample types via GHCL, scores retrieval candidates, and filters easy ones, while the expressive model trains exclusively on hard samples using a deeper, attention-based network.
• Figure 3: Model AUC vs. sample difficulty, with "Exposed" for exposed negatives and "Rank-$N$"/ "Prerank-$N$" for negatives ranked after position $N$; smaller $N$ implies higher difficulty.
• Figure 4: HAP's online deployment in Toutiao: a two-stage pre-ranking pipeline with a lightweight model for easy candidates and an expressive model for hard ones; samples are grouped and common features are shared for efficiency.
• Figure 5: Ablations of GHCL in HAP: removing GHCL leads to a notable AUC decrease across all test sets.