An Efficient Framework for Whole-Page Reranking via Single-Modal Supervision

TL;DR

SMAR tackles the high cost of whole-page reranking across heterogeneous modalities by distilling fine-grained signals from strong single-modal rankers. It introduces budget-aware annotation strategies (Top-P and Iso-label anchors) and two supervision modes (pairwise and listwise) to train a cross-modal reranker that preserves intra-modality ordering while aligning cross-modal preferences via cross-attention. Empirical results on industrial datasets Qilin and DuRank show a 70–90% reduction in page-level annotation costs with meaningful gains in NDCG, CTR, and engagement, corroborated by online A/B tests on Baidu Apps that report improvements in UV, IRQ, and next-day retention. The work demonstrates a scalable, real-world path to improved SERP quality by leveraging existing modal-specific expertise and targeted annotations.

Abstract

The whole-page reranking plays a critical role in shaping the user experience of search engines, which integrates retrieval results from multiple modalities, such as documents, images, videos, and LLM outputs. Existing methods mainly rely on large-scale human-annotated data, which is costly to obtain and time-consuming. This is because whole-page annotation is far more complex than single-modal: it requires assessing the entire result page while accounting for cross-modal relevance differences. Thus, how to improve whole-page reranking performance while reducing annotation costs is still a key challenge in optimizing search engine result pages(SERP). In this paper, we propose SMAR, a novel whole-page reranking framework that leverages strong Single-modal rankers to guide Modal-wise relevance Alignment for effective Reranking, using only limited whole-page annotation to outperform fully-annotated reranking models. Specifically, high-quality single-modal rankers are first trained on data specific to their respective modalities. Then, for each query, we select a subset of their outputs to construct candidate pages and perform human annotation at the page level. Finally, we train the whole-page reranker using these limited annotations and enforcing consistency with single-modal preferences to maintain ranking quality within each modality. Experiments on the Qilin and Baidu datasets demonstrate that SMAR reduces annotation costs by about 70-90\% while achieving significant ranking improvements compared to baselines. Further offline and online A/B testing on Baidu APPs also shows notable gains in standard ranking metrics as well as user experience indicators, fully validating the effectiveness and practical value of our approach in real-world search scenarios.

Figures (7)

• Figure 1: An example of heterogeneous retrieval and reranking on a mobile search interface. Given a user query "How to cook cola chicken", candidate results are recalled from multiple modality-specific queues, including LLM outputs, video content, and natural text.
• Figure 2: Imbalanced Upstream Model Scores Distributions Across Modals. Upstreammodel scores is a common feature score computed by the Single-Modal upstream models in Baidu search engine.
• Figure 3: Performance of Percentile bands on Qilin. The video queue and the natural result queue are sorted by upstream model scores. "Random" denotes randomly selecting 30% of the data from each of the two queues.
• Figure 4: Overall architecture of the proposed user–item interaction model. 1The encoder integrates query semantics with item representations, where a pretrained language model (PLM) encodes query–item textual signals and an item feature extractor incorporates auxiliary item information. 2User-specific features are independently processed by a user feature extractor to produce user embeddings, which are injected into the classifier as keys and values. 3The classifier employs cross-attention to capture fine-grained interactions between user embeddings and item representations, followed by normalization and multi-layer perceptron (MLP) modules. The head outputs the final relevance scores for candidate items.
• Figure 5: An example of binary search for iso-label anchors. Blue dashed boxes represent tie segments where items share the same label, and orange dashed boxes indicate virtual ties where no item in the natural queue has the same label as $v_2$, but natural items with both higher and lower labels exist, leading to the insertion of $v_2$ as a virtual tie.