Aligning Language Models for Versatile Text-based Item Retrieval

TL;DR

The paper tackles the inadequacy of general-purpose text embeddings for item retrieval by proposing an in-domain fine-tuning framework built on a ten-task dataset. It formalizes item retrieval with a query encoder over an item repository $\mathcal{T}$ and demonstrates substantial, task-wide gains when models are trained on in-domain data, including prominent improvements on the Xbox and Steam datasets. The authors show that fine-tuning yields strong in-domain performance, with notable improvements such as $Hit@5$ gains and better generalization in non-behavior-based tasks, and they validate practical impact through Chat-Rec in a one-turn conversational setting. The work provides a scalable path to align language models with item-level retrieval needs, significantly boosting both standalone retrieval and conversational recommender capabilities, with potential wide-ranging implications for search and commerce systems.

Abstract

This paper addresses the gap between general-purpose text embeddings and the specific demands of item retrieval tasks. We demonstrate the shortcomings of existing models in capturing the nuances necessary for zero-shot performance on item retrieval tasks. To overcome these limitations, we propose generate in-domain dataset from ten tasks tailored to unlocking models' representation ability for item retrieval. Our empirical studies demonstrate that fine-tuning embedding models on the dataset leads to remarkable improvements in a variety of retrieval tasks. We also illustrate the practical application of our refined model in a conversational setting, where it enhances the capabilities of LLM-based Recommender Agents like Chat-Rec. Our code is available at https://github.com/microsoft/RecAI.

Figures (1)

• Figure 1: Embedding any text for item retrieval.