ADORE: Autonomous Domain-Oriented Relevance Engine for E-commerce

TL;DR

ADORE tackles domain-specific relevance gaps in e-commerce search by integrating a rule-aware relevance discriminator with Chain-of-Thought reasoning, Kahneman-Tversky aligned training, adversarial data synthesis, and attribute-aware knowledge distillation. The framework automates annotation, data generation, and distillation to produce a cognizant, robust online model with explicit domain knowledge. Large-scale offline and online experiments on JD's system show consistent improvements over strong baselines in precision, F1, CTR, and revenue, validating the practical viability of cognitively aligned relevance modeling. This work proposes a resource-efficient paradigm for industrial relevance modeling that leverages LLM reasoning and targeted distillation to bridge semantic gaps and data scarcity.

Abstract

Relevance modeling in e-commerce search remains challenged by semantic gaps in term-matching methods (e.g., BM25) and neural models' reliance on the scarcity of domain-specific hard samples. We propose ADORE, a self-sustaining framework that synergizes three innovations: (1) A Rule-aware Relevance Discrimination module, where a Chain-of-Thought LLM generates intent-aligned training data, refined via Kahneman-Tversky Optimization (KTO) to align with user behavior; (2) An Error-type-aware Data Synthesis module that auto-generates adversarial examples to harden robustness; and (3) A Key-attribute-enhanced Knowledge Distillation module that injects domain-specific attribute hierarchies into a deployable student model. ADORE automates annotation, adversarial generation, and distillation, overcoming data scarcity while enhancing reasoning. Large-scale experiments and online A/B testing verify the effectiveness of ADORE. The framework establishes a new paradigm for resource-efficient, cognitively aligned relevance modeling in industrial applications.

Figures (1)

• Figure 1: The architecture of the Autonomous Domain-Oriented Relevance Engine.