Multi-task Retrieval for Knowledge-Intensive Tasks

TL;DR

The paper tackles the fragility of neural retrievers under out-of-domain conditions for knowledge-intensive tasks. It introduces a universal retrieval model trained jointly on eight KILT datasets, using a shared passage encoder and query encoder to learn universal representations. The authors demonstrate strong zero-shot and few-shot robustness and show improved downstream performance when integrated into existing systems, outperforming baselines like BM25 and comparable specialized retrievers. They further enhance training with adversarial confounder selection, and provide a practical, ready-to-use retrieval system with potential for multi-task industrial deployment.

Abstract

Retrieving relevant contexts from a large corpus is a crucial step for tasks such as open-domain question answering and fact checking. Although neural retrieval outperforms traditional methods like tf-idf and BM25, its performance degrades considerably when applied to out-of-domain data. Driven by the question of whether a neural retrieval model can be universal and perform robustly on a wide variety of problems, we propose a multi-task trained model. Our approach not only outperforms previous methods in the few-shot setting, but also rivals specialised neural retrievers, even when in-domain training data is abundant. With the help of our retriever, we improve existing models for downstream tasks and closely match or improve the state of the art on multiple benchmarks.

Figures (3)

• Figure 1: Training of DPR dpr, a bi-encoder model for open-domain question answering. Queries and passages are encoded as vectors, and retrieval is performed as a maximum inner product search.
• Figure 2: Two retrieval tasks performed by two fully-specialised models.
• Figure 3: Parameter sharing between neural retrievers.