Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigm

TL;DR

The paper tackles the semantic gap in dense retrieval by introducing Large Reasoning Embedding Model (LREM), which first reasons about a query via a chain-of-thought (CoT) and then produces a reasoning-augmented embedding for retrieval. It couples a data-construction pipeline that generates 75.06 million Query-CoT-Item triplets with a two-stage training regime: cold-start training using supervised fine-tuning and InfoNCE, followed by reinforcement learning using GRPO with format, length, and retrieval-accuracy rewards. Offline and online experiments show that LREM outperforms strong direct-embedding baselines, with notable gains on challenging query categories and a deployment on a major e-commerce platform since August 2025. The approach demonstrates that explicit reasoning can bridge semantic gaps in embeddings, offering a practical path to more intelligent dense retrievers in large-scale settings.

Abstract

In modern e-commerce search systems, dense retrieval has become an indispensable component. By computing similarities between query and item (product) embeddings, it efficiently selects candidate products from large-scale repositories. With the breakthroughs in large language models (LLMs), mainstream embedding models have gradually shifted from BERT to LLMs for more accurate text modeling. However, these models still adopt direct-embedding methods, and the semantic accuracy of embeddings remains inadequate. Therefore, contrastive learning is heavily employed to achieve tight semantic alignment between positive pairs. Consequently, such models tend to capture statistical co-occurrence patterns in the training data, biasing them toward shallow lexical and semantic matches. For difficult queries exhibiting notable lexical disparity from target items, the performance degrades significantly. In this work, we propose the Large Reasoning Embedding Model (LREM), which novelly integrates reasoning processes into representation learning. For difficult queries, LREM first conducts reasoning to achieve a deep understanding of the original query, and then produces a reasoning-augmented query embedding for retrieval. This reasoning process effectively bridges the semantic gap between original queries and target items, significantly improving retrieval accuracy. Specifically, we adopt a two-stage training process: the first stage optimizes the LLM on carefully curated Query-CoT-Item triplets with SFT and InfoNCE losses to establish preliminary reasoning and embedding capabilities, and the second stage further refines the reasoning trajectories via reinforcement learning (RL). Extensive offline and online experiments validate the effectiveness of LREM, leading to its deployment on China's largest e-commerce platform since August 2025.

Figures (5)

• Figure 1: Comparison between traditional direct-embedding and the proposed reasoning-then-embedding dense retriever (LREM). LREM leverages reasoning to enable deep query understanding and accurate embeddings, overcoming the superficiality of direct-embedding methods.
• Figure 2: (1) Data Construction: An LLM generates keyword-based CoTs for each query. By comparing the retrieval results of a traditional dense retriever with and without the CoT, we discard queries where the CoT provides no gains, and filter truly relevant items via an advanced relevance model for the remaining queries. (2) Cold Start: LREM is trained on Query-CoT-Item triplets, where the SFT loss optimizes LREM's reasoning process and the InfoNCE loss aligns the reasoning‑augmented query embedding with the item embedding. (3) Reinforcement Learning: LREM is trained on Query-Item pairs, where the GRPO loss encourages exploration of superior reasoning trajectories under the guidance of a reward system, and the InfoNCE loss concurrently aligns the embeddings. Note that the same LREM is applied to both the query-side and the item-side embedding.
• Figure 3: Representative examples from the four query categories. Compared with the direct-embedding method Qwen2.5 (Uni-Attn. Last), which performs only superficial lexical matching, LREM adopts a novel reasoning-then-embedding paradigm, enabling accurate query semantic understanding and target item retrieval.
• Figure 4: Comparison of generated CoT and retrieval results between LREM (Cold Start) and LREM (Cold Start + RL).
• Figure 5: Retrieval performance across varying CoT lengths generated by LREM.