Do Reasoning Models Enhance Embedding Models?

TL;DR

The paper asks whether RLVR-enhanced reasoning backbones yield better text embeddings. Using a unified HRSA framework, it shows RLVR largely preserves global latent-manifold geometry and linear readouts but induces irreversible local geometry changes, with contrastive learning driving rapid manifold realignment that nullifies potential advantages. The key finding is that RLVR improves the trajectory through an existing semantic landscape rather than rewriting it, suggesting that geometry-aware regularization during supervised fine-tuning could mimic RLVR’s local-geometry effects without sacrificing global structure. This work provides a diagnostic toolkit for representation learning and has implications for designing embedding pipelines that balance stability of global semantics with flexible local adaptations.

Abstract

State-of-the-art embedding models are increasingly derived from decoder-only Large Language Model (LLM) backbones adapted via contrastive learning. Given the emergence of reasoning models trained via Reinforcement Learning with Verifiable Rewards (RLVR), a natural question arises: do enhanced reasoning translate to superior semantic representations when these models serve as embedding initializations? Contrary to expectation, our evaluation on MTEB and BRIGHT reveals a **null effect**: embedding models initialized from RLVR-tuned backbones yield no consistent performance advantage over their base counterparts when subjected to identical training recipes. To unpack this paradox, we introduce **H**ierarchical **R**epresentation **S**imilarity **A**nalysis (HRSA), a framework that decomposes similarity across representation, geometry, and function levels. HRSA reveals that while RLVR induces irreversible latent manifold's local geometry reorganization and reversible coordinate basis drift, it preserves the global manifold geometry and linear readout. Consequently, subsequent contrastive learning drives strong alignment between base- and reasoning-initialized models, a phenomenon we term **Manifold Realignment**. Empirically, our findings suggest that unlike Supervised Fine-Tuning (SFT), RLVR optimizes trajectories within an existing semantic landscape rather than fundamentally restructuring the landscape itself.

Figures (13)

• Figure 1: Latent manifold and model relationships.CL, SFT, and RLVR denote Contrastive Learning, Supervised Fine-Tuning, and Reinforcement Learning with Verifiable Rewards, respectively. $\textbf{z}$ indicates the representations of the corresponding models. Suffix "-Emb" is added to the model name to indicate the embedding model. We demonstrate the ideas of similar and dissimilar representations of RLVR-tuned pairs and SFT-tuned pairs, respectively.
• Figure 2: The overview of HRSA.
• Figure 3: Heatmap of Dimension-Wise Correlation (left) and Linear CKA (right).Columns: SFT vs. RLVR. Rows: $\mathcal{M}_{base}$ vs. $\mathcal{M}_{reason}$ and $\mathcal{M}_{base}^\textit{Emb}$ vs. $\mathcal{M}_{reason}^\textit{Emb}$.
• Figure 4: Cross-Model Linear Probe Results. For each dataset split (train, dev, test), the left bar corresponds to $\mathcal{M}_{base}$ (or $\mathcal{M}_{base}^\textit{Emb}$) and the right bar corresponds to $\mathcal{M}_{reason}$ (or $\mathcal{M}_{reason}^\textit{Emb}$). The linear probe is trained on $\mathcal{M}_{base}$ (or $\mathcal{M}_{base}^\textit{Emb}$ in embedding model analysis) representations and evaluated on both models. The smaller the $\Delta$, the stronger the cross-model linear probe transfer.
• Figure 5: The training dynamics of the embedding model pairs DS-$Emb$ vs ProRL-$Emb$. Step 0 indicates LLM backbones, and step 781 indicates the final checkpoint of the embedding models.

Theorems & Definitions (18)

• Definition 1: Representation-Level Analysis
• Proposition 1
• proof
• Proposition 2
• proof
• Remark
• Definition 2: Geometry-Level Analysis
• Proposition 3
• proof
• Proposition 4