Search-Adaptor: Embedding Customization for Information Retrieval

TL;DR

Search-Adaptor addresses the challenge of improving information retrieval by customizing pre-trained LLM embeddings without access to model weights. It introduces a lightweight, shared adapter with skip connections, a training-time query predictor, and a triad of losses (ranking, recovery, and prediction) to robustly adapt embeddings while preserving generalization. The approach yields consistent gains across API-based and open LLMs, English and multilingual data, and even multimodal and tool-retrieval scenarios, demonstrating strong data-efficiency and practical deployability. By enabling embedding customization with minimal access requirements, the method promotes model distillation-like benefits and broad applicability in real-world retrieval systems.

Abstract

Embeddings extracted by pre-trained Large Language Models (LLMs) have significant potential to improve information retrieval and search. Beyond the zero-shot setup in which they are being conventionally used, being able to take advantage of the information from the relevant query-corpus paired data can further boost the LLM capabilities. In this paper, we propose a novel method, Search-Adaptor, for customizing LLMs for information retrieval in an efficient and robust way. Search-Adaptor modifies the embeddings generated by pre-trained LLMs, and can be integrated with any LLM, including those only available via prediction APIs. On multiple English, multilingual, and multimodal retrieval datasets, we show consistent and significant performance benefits for Search-Adaptor -- e.g., more than 5% improvements for Google Embedding APIs in nDCG@10 averaged over 14 BEIR datasets.

Figures (7)

• Figure 1: Search-Adaptor modifies the pre-trained LLM embeddings by learning from paired query-document data, yielding significantly improved retrieval performance on target tasks. Note that Search-Adaptor does not require access to the LLM weights or gradients and can even be applied to the LLMs that are only accessible via prediction APIs.
• Figure 2: Block diagram of Search-Adaptor. Grey colored blocks are fixed components (e.g., a text embedding API); blue-colored blocks are additional trainable building blocks; and red-colored blocks are for loss computations. At inference, only query and corpus adapters are utilized and the query predictor can be discarded.
• Figure 3: Performance improvements with Search-Adaptor on top of Google's LLM based embedding APIs (gecko@latest, 768 dimensions) for 14 BEIR datasets.
• Figure 4: Performance improvements with Search-Adaptor on top of the Google's LLM based embedding APIs (gecko-multilingual@latest, 768 dimensions) for 17 multilingual MIRACL datasets.
• Figure 5: Cosine similarity score distributions before and after Search-Adaptor.