Align then Train: Efficient Retrieval Adapter Learning

Abstract

Dense retrieval systems increasingly need to handle complex queries. In many realistic settings, users express intent through long instructions or task-specific descriptions, while target documents remain relatively simple and static. This asymmetry creates a retrieval mismatch: understanding queries may require strong reasoning and instruction-following, whereas efficient document indexing favors lightweight encoders. Existing retrieval systems often address this mismatch by directly improving the embedding model, but fine-tuning large embedding models to better follow such instructions is computationally expensive, memory-intensive, and operationally burdensome. To address this challenge, we propose Efficient Retrieval Adapter (ERA), a label-efficient framework that trains retrieval adapters in two stages: self-supervised alignment and supervised adaptation. Inspired by the pre-training and supervised fine-tuning stages of LLMs, ERA first aligns the embedding spaces of a large query embedder and a lightweight document embedder, and then uses limited labeled data to adapt the query-side representation, bridging both the representation gap between embedding models and the semantic gap between complex queries and simple documents without re-indexing the corpus. Experiments on the MAIR benchmark, spanning 126 retrieval tasks across 6 domains, show that ERA improves retrieval in low-label settings, outperforms methods that rely on larger amounts of labeled data, and effectively combines stronger query embedders with weaker document embedders across domains.

Figures (9)

• Figure 1: Left: Existing retrieval systems often treat the query/document sides symmetrically, using the same model. Right: ERA decouples retrieval adapter training into two stages: a self-supervised alignment stage that bridges the representation gap between different embedding models, and a supervised adaptation stage that captures the semantic nuances between complex queries and simple documents without re-indexing.
• Figure 2: The overview of Efficient Retrieval Adapter.
• Figure 3: nDCG@10 of zero-shot, symmetric ERA and asymmetric ERA retrieval, where we use Qwen3-8B as the query embedder.
• Figure 4: nDCG@10 of zero-shot, ERA w/o adaptation, Embedding Adapter, and ERA retrieval at varying train ratios, where we use Qwen3-8B as the query embedder.
• Figure 5: Domain generality analysis using Qwen3-8B as the query embedder and Qwen3-0.6B as the document embedder, trained on 20% of labeled query-document pairs.