A Representation Sharpening Framework for Zero Shot Dense Retrieval

TL;DR

The paper tackles zero-shot dense retrieval when a target corpus lacks relevance-labelled queries. It proposes a training-free representation sharpening framework that augments document embeddings with contrastive queries generated in a many-to-many paradigm, weighted by query similarity, to sharpen representations without retraining. Extensive experiments across BEIR, BRIGHT, and MIRACL demonstrate consistent gains, including state-of-the-art results on BRIGHT and substantial multilingual improvements, with both inference-time and indexing-time variants to balance performance and cost. The approach is complementary to prior methods and highlights practical deployment options that avoid additional inference overhead while boosting retrieval quality.

Abstract

Zero-shot dense retrieval is a challenging setting where a document corpus is provided without relevant queries, necessitating a reliance on pretrained dense retrievers (DRs). However, since these DRs are not trained on the target corpus, they struggle to represent semantic differences between similar documents. To address this failing, we introduce a training-free representation sharpening framework that augments a document's representation with information that helps differentiate it from similar documents in the corpus. On over twenty datasets spanning multiple languages, the representation sharpening framework proves consistently superior to traditional retrieval, setting a new state-of-the-art on the BRIGHT benchmark. We show that representation sharpening is compatible with prior approaches to zero-shot dense retrieval and consistently improves their performance. Finally, we address the performance-cost tradeoff presented by our framework and devise an indexing-time approximation that preserves the majority of our performance gains over traditional retrieval, yet suffers no additional inference-time cost.

Figures (4)

• Figure 1: Left: Comparison of the traditional dense retrieval pipeline and our proposed framework for representation sharpening. During the indexing phase, we generate contrastive queries and store them along with a document's embedding as metadata. During inference, the inference query is used to determine weights that signify which contrastive queries are most relevant and we aggregate their embeddings using these weights. We use this aggregated embedding to augment the document representation before computing relevance. Right: Visualization of how sharpening improves retrieval. The contrastive queries help refine the embeddings and accentuate the most unique aspects of each document, hence enabling more precise differentiation between similar documents.
• Figure 2: Performance improvement on SciFact when varying the underlying LM. Performance scales with model size, however Qwen3-8B still delivers a noteably performance boost, showing that even open-weight models on the 8B parameter scale can be used to generate effective contrastive queries.
• Figure 3: Performance when varying $\alpha$ on E5-Mistral. Increasing $\alpha$ from 0 to 1 always leads to improvements and performance declines after peaking in the $\alpha\in[1, 1.5]$ range. The best value of $\alpha$ is never the value used in our experiments ($\alpha=1$), suggesting that hyperparameter tuning can provide further gains.
• Figure 4: Performance when varying the average number of queries used per document on E5-Mistral. Using more queries increases performance, with the majority of the improvement occurs with the first $10$ queries.