Thinking Broad, Acting Fast: Latent Reasoning Distillation from Multi-Perspective Chain-of-Thought for E-Commerce Relevance

TL;DR

This work tackles the challenge of delivering accurate e-commerce relevance with real-time latency by enabling rich reasoning during training and fast inference at deployment. It introduces Multi-Perspective Chain-of-Thought (MPCoT) to generate diverse rationales from user intent, structured analysis, and business rules, paired with Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to produce a robust teacher. It then distills this reasoning into a lightweight cross-encoder via Latent Reasoning Knowledge Distillation (LRKD), preserving reasoning semantics in the embedding space without generating text during inference. Offline evaluations on multilingual AliExpress and ESCI datasets show consistent improvements over single-perspective CoT and prior distillation methods, while online A/B testing confirms gains in Revenue Per Mille (RPM), Click-Through Rate (CTR), and relevance satisfaction. Overall, the framework demonstrates a practical path to leverage LLM reasoning for scalable, interpretable, and high-performing e-commerce relevance models.

Abstract

Effective relevance modeling is crucial for e-commerce search, as it aligns search results with user intent and enhances customer experience. Recent work has leveraged large language models (LLMs) to address the limitations of traditional relevance models, especially for long-tail and ambiguous queries. By incorporating Chain-of-Thought (CoT) reasoning, these approaches improve both accuracy and interpretability through multi-step reasoning. However, two key limitations remain: (1) most existing approaches rely on single-perspective CoT reasoning, which fails to capture the multifaceted nature of e-commerce relevance (e.g., user intent vs. attribute-level matching vs. business-specific rules); and (2) although CoT-enhanced LLM's offer rich reasoning capabilities, their high inference latency necessitates knowledge distillation for real-time deployment, yet current distillation methods discard the CoT rationale structure at inference, using it as a transient auxiliary signal and forfeiting its reasoning utility. To address these challenges, we propose a novel framework that better exploits CoT semantics throughout the optimization pipeline. Specifically, the teacher model leverages Multi-Perspective CoT (MPCoT) to generate diverse rationales and combines Supervised Fine-Tuning (SFT) with Direct Preference Optimization (DPO) to construct a more robust reasoner. For distillation, we introduce Latent Reasoning Knowledge Distillation (LRKD), which endows a student model with a lightweight inference-time latent reasoning extractor, allowing efficient and low-latency internalization of the LLM's sophisticated reasoning capabilities. Evaluated in offline experiments and online A/B tests on an e-commerce search advertising platform serving tens of millions of users daily, our method delivers significant offline gains, showing clear benefits in both commercial performance and user experience.

Figures (5)

• Figure 1: From Single to Multiple Perspectives in Reasoning
• Figure 2: Framework overview of MPCoT teacher training (top) and LRKD student distillation and deployment (bottom).
• Figure 3: Performance comparison of Chain-of-Thought (CoT) generated from different perspectives on the AliExpress dataset.
• Figure 4: Probe F1 scores for top non-trivial CoT keywords
• Figure 5: Case studies from the AliExpress Dataset.