Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrieval

Jianlyu Chen, Junwei Lan, Chaofan Li, Defu Lian, Zheng Liu

TL;DR

ReasonEmbed targets reasoning-intensive document retrieval by generating high-quality synthetic training data with ReMixer and training with a self-adaptive RI-InfoNCE loss (Redapter). The three-stage data synthesis and the RI-based weighting enable the model to capture complex query–document relations across multiple backbones. Empirical results on BRIGHT and R2MED show state-of-the-art performance and clear gains from data synthesis, annotation quality, and reasoning-aware training. The work also emphasizes open science by releasing resources to accelerate progress in reasoning-aware IR.

Abstract

In this paper, we introduce ReasonEmbed, a novel text embedding model developed for reasoning-intensive document retrieval. Our work includes three key technical contributions. First, we propose ReMixer, a new data synthesis method that overcomes the triviality problem prevalent in previous synthetic datasets, enabling large-scale production of 82K high-quality training samples. Second, we design Redapter, a self-adaptive learning algorithm that dynamically adjusts training each sample's weight based on its reasoning intensity. This allows the model to effectively capture the complex semantic relationships between queries and documents. Third, we implement ReasonEmbed across multiple backbones of varying sizes, all of which achieve superior performance on reasoning-intensive retrieval tasks. Notably, our ReasonEmbed-Qwen3-8B model offers a record-high nDCG@10 score of 38.1 on the BRIGHT benchmark, which significantly outperforms existing text embedding models. We will fully open-source our created resources in ReasonEmbed to push forward the research advancement in this field.

ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrieval

TL;DR

ReasonEmbed targets reasoning-intensive document retrieval by generating high-quality synthetic training data with ReMixer and training with a self-adaptive RI-InfoNCE loss (Redapter). The three-stage data synthesis and the RI-based weighting enable the model to capture complex query–document relations across multiple backbones. Empirical results on BRIGHT and R2MED show state-of-the-art performance and clear gains from data synthesis, annotation quality, and reasoning-aware training. The work also emphasizes open science by releasing resources to accelerate progress in reasoning-aware IR.

Abstract

In this paper, we introduce ReasonEmbed, a novel text embedding model developed for reasoning-intensive document retrieval. Our work includes three key technical contributions. First, we propose ReMixer, a new data synthesis method that overcomes the triviality problem prevalent in previous synthetic datasets, enabling large-scale production of 82K high-quality training samples. Second, we design Redapter, a self-adaptive learning algorithm that dynamically adjusts training each sample's weight based on its reasoning intensity. This allows the model to effectively capture the complex semantic relationships between queries and documents. Third, we implement ReasonEmbed across multiple backbones of varying sizes, all of which achieve superior performance on reasoning-intensive retrieval tasks. Notably, our ReasonEmbed-Qwen3-8B model offers a record-high nDCG@10 score of 38.1 on the BRIGHT benchmark, which significantly outperforms existing text embedding models. We will fully open-source our created resources in ReasonEmbed to push forward the research advancement in this field.

Paper Structure

This paper contains 31 sections, 3 equations, 3 figures, 20 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Synthesis: ReMixer
  4. Query Generation
  5. Candidate Mining
  6. Relevance Annotation
  7. Synthesization Result
  8. Training Method: Redapter
  9. Reasoning Intensity
  10. Self-Adaptive Learning
  11. Experiments
  12. Setup
  13. Main Results
  14. Ablation Study
  15. Candidate Mining
  16. ...and 16 more sections

Figures (3)

  • Figure 1: The three-stage data synthesis workflow of ReMixer. The full prompts used in the data synthesis process are available in Appendix \ref{['sec:appendix:data_synthesis']}.
  • Figure 2: Impact of synthetic data size on retrieval accuracy (using basic contrastive learning for simplicity).
  • Figure 3: Data contamination analysis results (the computed max weighted Jaccard similarity) between our synthetic dataset and the testing data in BRIGHT and R2MED.