Toward Optimal Search and Retrieval for RAG

TL;DR

This work works towards the goal of understanding how retrievers can be optimized for RAG pipelines for common tasks such as Question Answering, and uncovers a number of insights useful to practitioners developing high-performance RAG pipelines.

Abstract

Retrieval-augmented generation (RAG) is a promising method for addressing some of the memory-related challenges associated with Large Language Models (LLMs). Two separate systems form the RAG pipeline, the retriever and the reader, and the impact of each on downstream task performance is not well-understood. Here, we work towards the goal of understanding how retrievers can be optimized for RAG pipelines for common tasks such as Question Answering (QA). We conduct experiments focused on the relationship between retrieval and RAG performance on QA and attributed QA and unveil a number of insights useful to practitioners developing high-performance RAG pipelines. For example, lowering search accuracy has minor implications for RAG performance while potentially increasing retrieval speed and memory efficiency.

Figures (12)

• Figure 1: Example prompts for QA (left) and attributed QA (right, following Gao_Yen_Yu_Chen_2023).
• Figure 2: Correctness achieved by Mistral with various numbers of documents retrieved with BGE-base and ColBERT. Optimal performance is observed with $k=10$ or $20$.
• Figure 3: The per-query relationship between the number of gold documents included in the prompt and the QA accuracy achieved with Mistral on ASQA. Including just one gold document significantly improves accuracy. There is a correlation between the number of gold documents and EM Recall.
• Figure 4: Gold document recall (left) has a greater impact on RAG QA performance compared to search recall (right). RAG pipeline uses Mistral and BGE-base. Shaded bar is ceiling performance using all gold documents per query. Error bars are 95% bootstrap confidence intervals.
• Figure 5: ASQA Mistral performance after injecting noisy documents from various percentiles of similarity to the query. Adding noisy documents from all percentiles degrades QA correctness.