SERM: Self-Evolving Relevance Model with Agent-Driven Learning from Massive Query Streams

TL;DR

SERM introduces a self-evolving framework for search relevance that operates over massive, evolving query streams. It combines a Multi-Agent Sample Miner to detect distribution shifts and sample informative data with a Multi-Agent Relevance Annotator to produce reliable labels through inner- and inter-agent agreement, addressing sparse informative samples and unreliable pseudo-labels. Across large-scale industrial data, SERM yields stable, significant gains over traditional fine-tuning and self-training, with notable online improvements in retention and user satisfaction and effective distillation for latency-constrained deployments. The work demonstrates robust performance across multilingual settings and provides a practical pathway for deploying continually adapting relevance models in real-world systems.

Abstract

Due to the dynamically evolving nature of real-world query streams, relevance models struggle to generalize to practical search scenarios. A sophisticated solution is self-evolution techniques. However, in large-scale industrial settings with massive query streams, this technique faces two challenges: (1) informative samples are often sparse and difficult to identify, and (2) pseudo-labels generated by the current model could be unreliable. To address these challenges, in this work, we propose a Self-Evolving Relevance Model approach (SERM), which comprises two complementary multi-agent modules: a multi-agent sample miner, designed to detect distributional shifts and identify informative training samples, and a multi-agent relevance annotator, which provides reliable labels through a two-level agreement framework. We evaluate SERM in a large-scale industrial setting, which serves billions of user requests daily. Experimental results demonstrate that SERM can achieve significant performance gains through iterative self-evolution, as validated by extensive offline multilingual evaluations and online testing.

Figures (6)

• Figure 1: (a) Learning from Static Data: The conventional training recipe is to first apply continual pre-training and then perform supervised fine-tuning on static labeled data. (b) Learning from Massive Query Streams: The proposed SERM uses a multi-agent sample miner to identify informative samples and a multi-agent relevance annotator to generate reliable labels, enabling continuous model evolution with massive query streams.
• Figure 2: Side-by-side manual evaluation results comparing our proposed SERM with the baseline.
• Figure 3: Performance on different thresholds.
• Figure 4: Length distribution of document datasets and queries used in our experiments.
• Figure 5: Template used for training our generative relevance models.