Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Multilingual Search Relevance Modeling in E-Commerce via LLM Mixture-of-Experts

Ye Liu, Xu Chen, Wuji Chen, Mang Li

TL;DR

The paper tackles multilingual e-commerce search relevance under cross-country distribution shifts by introducing a sparse LLM-based Mixture-of-Experts framework. It leverages the complementary strengths of multiple fine-tuned LLMs through a dynamic router that selects top-k experts and concatenates their projected embeddings for final relevance prediction, with end-to-end hard routing and a load-balancing term $L_{ ext{LB}}$ as the default. An offline batch inference pipeline further enhances efficiency via sparsity-aware computation, parallel execution, and scalable scheduling. Experiments across six Southeast Asian markets show superior AUC (92.49%) and improved throughput (QPS 13.72) over strong dense baselines and full fusion, demonstrating practical cost-effectiveness for real-world deployment. The work highlights the value of heterogeneous expert fusion for multilingual relevance and outlines a viable path to production-scale inference with reduced GPU-hour consumption by up to 35% and higher query throughput.

Abstract

In e-commerce platforms, search relevance directly influences both user experience and merchant revenue. In multi-country deployments, diverse linguistic, cultural, and product catalog contexts introduce significant distribution shifts, posing substantial challenges to relevance modeling. Existing approaches typically enhance the reasoning or multilingual abilities of a single monolithic model, yet they remain limited by data diversity, coverage gaps, and high inference costs in heterogeneous environments. Our empirical analysis reveals that different LLM base models exhibit complementary strengths across languages and regions, motivating an expert-based architecture. We propose a scalable LLM-based Mixture-of-Experts (MoE) framework that dynamically routes queries to specialized experts and fuses their embeddings through concatenation. Among rule-based, pseudo-label-based, and fully end-to-end strategies, end-to-end hard routing with concatenation offers the best balance of effectiveness and efficiency. To mitigate inference overhead, we further develop an engineering-optimized offline batch pipeline with resource-efficient scheduling, which hides memory latency, improves GPU utilization, and reduces GPU-hour consumption by up to 35% compared with synchronous execution. On datasets spanning six Southeast Asian markets, our MoE improves AUC by 0.72 percentage points over a dense baseline with the same active parameters. Meanwhile, the optimized pipeline achieves 27.6 queries per second (QPS), a 9% throughput improvement. These results demonstrate superior multilingual relevance and efficiency, delivering strong cost-effectiveness for real-world e-commerce search systems.

Efficient Multilingual Search Relevance Modeling in E-Commerce via LLM Mixture-of-Experts

TL;DR

The paper tackles multilingual e-commerce search relevance under cross-country distribution shifts by introducing a sparse LLM-based Mixture-of-Experts framework. It leverages the complementary strengths of multiple fine-tuned LLMs through a dynamic router that selects top-k experts and concatenates their projected embeddings for final relevance prediction, with end-to-end hard routing and a load-balancing term as the default. An offline batch inference pipeline further enhances efficiency via sparsity-aware computation, parallel execution, and scalable scheduling. Experiments across six Southeast Asian markets show superior AUC (92.49%) and improved throughput (QPS 13.72) over strong dense baselines and full fusion, demonstrating practical cost-effectiveness for real-world deployment. The work highlights the value of heterogeneous expert fusion for multilingual relevance and outlines a viable path to production-scale inference with reduced GPU-hour consumption by up to 35% and higher query throughput.

Abstract

In e-commerce platforms, search relevance directly influences both user experience and merchant revenue. In multi-country deployments, diverse linguistic, cultural, and product catalog contexts introduce significant distribution shifts, posing substantial challenges to relevance modeling. Existing approaches typically enhance the reasoning or multilingual abilities of a single monolithic model, yet they remain limited by data diversity, coverage gaps, and high inference costs in heterogeneous environments. Our empirical analysis reveals that different LLM base models exhibit complementary strengths across languages and regions, motivating an expert-based architecture. We propose a scalable LLM-based Mixture-of-Experts (MoE) framework that dynamically routes queries to specialized experts and fuses their embeddings through concatenation. Among rule-based, pseudo-label-based, and fully end-to-end strategies, end-to-end hard routing with concatenation offers the best balance of effectiveness and efficiency. To mitigate inference overhead, we further develop an engineering-optimized offline batch pipeline with resource-efficient scheduling, which hides memory latency, improves GPU utilization, and reduces GPU-hour consumption by up to 35% compared with synchronous execution. On datasets spanning six Southeast Asian markets, our MoE improves AUC by 0.72 percentage points over a dense baseline with the same active parameters. Meanwhile, the optimized pipeline achieves 27.6 queries per second (QPS), a 9% throughput improvement. These results demonstrate superior multilingual relevance and efficiency, delivering strong cost-effectiveness for real-world e-commerce search systems.
Paper Structure (11 sections, 5 equations, 2 figures, 2 tables)

This paper contains 11 sections, 5 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. LLM-based Mixture-of-Experts Architecture
  4. Offline Batch Inference with Resource-Efficient Scheduling
  5. Experiments
  6. Dataset and Evaluation Metrics
  7. Experimental Settings
  8. Main Results
  9. Ablation Studies
  10. Online Deployment
  11. Conclusion

Figures (2)

  • Figure 1: Radar chart showing AUC of three base models across multiple country–language tasks.
  • Figure 2: Overview of the proposed sparsely-activated Mixture-of-Experts (MoE) framework for multilingual e-commerce search. The routing module dynamically selects top-k LLM experts based on query, nation, and item title; their hidden representations are projected and concatenated in the fusion module for relevance prediction. The bottom part shows the architectures of the routing and fusion modules, including soft and hard routing strategies.