Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Better Search with Domain-Aware Text Embeddings for C2C Marketplaces

Andre Rusli, Miao Cao, Shoma Ishimoto, Sho Akiyama, Max Frenzel

TL;DR

Mercari investigates domain-aware Japanese text embeddings to improve search in C2C marketplaces, addressing short, ambiguous queries and noisy listings. The authors fine-tune a text encoder with Multiple Negatives Ranking using purchase-driven query-title pairs, applying role-specific prefixes and Matryoshka Representation Learning to produce compact, truncation-robust 32-dimensional embeddings. Offline evaluations show consistent gains over a strong baseline and significant improvements when replacing PCA with Matryoshka truncation; public STS results indicate graceful degradation under truncation. Online A/B experiments demonstrate revenue and search efficiency gains, validating the approach for scalable, LLM-era search experiences in production.

Abstract

Consumer-to-consumer (C2C) marketplaces pose distinct retrieval challenges: short, ambiguous queries; noisy, user-generated listings; and strict production constraints. This paper reports our experiment to build a domain-aware Japanese text-embedding approach to improve the quality of search at Mercari, Japan's largest C2C marketplace. We experimented with fine-tuning on purchase-driven query-title pairs, using role-specific prefixes to model query-item asymmetry. To meet production constraints, we apply Matryoshka Representation Learning to obtain compact, truncation-robust embeddings. Offline evaluation on historical search logs shows consistent gains over a strong generic encoder, with particularly large improvements when replacing PCA compression with Matryoshka truncation. A manual assessment further highlights better handling of proper nouns, marketplace-specific semantics, and term-importance alignment. Additionally, an initial online A/B test demonstrates statistically significant improvements in revenue per user and search-flow efficiency, with transaction frequency maintained. Results show that domain-aware embeddings improve relevance and efficiency at scale and form a practical foundation for richer LLM-era search experiences.

Towards Better Search with Domain-Aware Text Embeddings for C2C Marketplaces

TL;DR

Mercari investigates domain-aware Japanese text embeddings to improve search in C2C marketplaces, addressing short, ambiguous queries and noisy listings. The authors fine-tune a text encoder with Multiple Negatives Ranking using purchase-driven query-title pairs, applying role-specific prefixes and Matryoshka Representation Learning to produce compact, truncation-robust 32-dimensional embeddings. Offline evaluations show consistent gains over a strong baseline and significant improvements when replacing PCA with Matryoshka truncation; public STS results indicate graceful degradation under truncation. Online A/B experiments demonstrate revenue and search efficiency gains, validating the approach for scalable, LLM-era search experiences in production.

Abstract

Consumer-to-consumer (C2C) marketplaces pose distinct retrieval challenges: short, ambiguous queries; noisy, user-generated listings; and strict production constraints. This paper reports our experiment to build a domain-aware Japanese text-embedding approach to improve the quality of search at Mercari, Japan's largest C2C marketplace. We experimented with fine-tuning on purchase-driven query-title pairs, using role-specific prefixes to model query-item asymmetry. To meet production constraints, we apply Matryoshka Representation Learning to obtain compact, truncation-robust embeddings. Offline evaluation on historical search logs shows consistent gains over a strong generic encoder, with particularly large improvements when replacing PCA compression with Matryoshka truncation. A manual assessment further highlights better handling of proper nouns, marketplace-specific semantics, and term-importance alignment. Additionally, an initial online A/B test demonstrates statistically significant improvements in revenue per user and search-flow efficiency, with transaction frequency maintained. Results show that domain-aware embeddings improve relevance and efficiency at scale and form a practical foundation for richer LLM-era search experiences.
Paper Structure (20 sections, 1 equation, 1 figure, 5 tables)

This paper contains 20 sections, 1 equation, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Previous Work
  3. Pre-Trained Query-Item Embeddings for Search Re-Ranking
  4. Known Limitations
  5. Methods
  6. Training Setup
  7. Loss Function and Dataset
  8. Query-Item Prefixes
  9. Dimensionality Reduction
  10. Offline Evaluation Methods
  11. Experiments
  12. Manual Qualitative Assessments
  13. Quantitative Evaluation on Past Search Logs
  14. Public Japanese STS benchmark
  15. Online A/B Test
  16. ...and 5 more sections

Figures (1)

  • Figure 1: Production serving architecture for the A/B test. Control and Treatment differ only in the embedding model routed via Triton; all other components are identical. Solid lines indicate data flow, dashed lines indicate API calls between services.