Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EnrichIndex: Using LLMs to Enrich Retrieval Indices Offline

Peter Baile Chen, Tomer Wolfson, Michael Cafarella, Dan Roth

TL;DR

EnrichIndex addresses the high online cost of LLM-based retrieval by performing a single offline pass over the corpus to enrich each object with four representations: the original content, a summary, its purpose, and QA pairs. During online retrieval, a weighted combination of similarities across these representations estimates query relevance, reducing the need for costly online document expansion. Across five datasets spanning documents and tables, EnrichIndex yields substantial gains in recall@10 and NDCG@10 while dramatically cutting online LLM token usage (up to 293.3x fewer tokens). The approach also complements online re-rankers, enabling stronger end-to-end retrieval with lower latency and cost, and provides detailed analyses of enrichment contributions across domains.

Abstract

Existing information retrieval systems excel in cases where the language of target documents closely matches that of the user query. However, real-world retrieval systems are often required to implicitly reason whether a document is relevant. For example, when retrieving technical texts or tables, their relevance to the user query may be implied through a particular jargon or structure, rather than explicitly expressed in their content. Large language models (LLMs) hold great potential in identifying such implied relevance by leveraging their reasoning skills. Nevertheless, current LLM-augmented retrieval is hindered by high latency and computation cost, as the LLM typically computes the query-document relevance online, for every query anew. To tackle this issue we introduce EnrichIndex, a retrieval approach which instead uses the LLM offline to build semantically-enriched retrieval indices, by performing a single pass over all documents in the retrieval corpus once during ingestion time. Furthermore, the semantically-enriched indices can complement existing online retrieval approaches, boosting the performance of LLM re-rankers. We evaluated EnrichIndex on five retrieval tasks, involving passages and tables, and found that it outperforms strong online LLM-based retrieval systems, with an average improvement of 11.7 points in recall @ 10 and 10.6 points in NDCG @ 10 compared to strong baselines. In terms of online calls to the LLM, it processes 293.3 times fewer tokens which greatly reduces the online latency and cost. Overall, EnrichIndex is an effective way to build better retrieval indices offline by leveraging the strong reasoning skills of LLMs.

EnrichIndex: Using LLMs to Enrich Retrieval Indices Offline

TL;DR

EnrichIndex addresses the high online cost of LLM-based retrieval by performing a single offline pass over the corpus to enrich each object with four representations: the original content, a summary, its purpose, and QA pairs. During online retrieval, a weighted combination of similarities across these representations estimates query relevance, reducing the need for costly online document expansion. Across five datasets spanning documents and tables, EnrichIndex yields substantial gains in recall@10 and NDCG@10 while dramatically cutting online LLM token usage (up to 293.3x fewer tokens). The approach also complements online re-rankers, enabling stronger end-to-end retrieval with lower latency and cost, and provides detailed analyses of enrichment contributions across domains.

Abstract

Existing information retrieval systems excel in cases where the language of target documents closely matches that of the user query. However, real-world retrieval systems are often required to implicitly reason whether a document is relevant. For example, when retrieving technical texts or tables, their relevance to the user query may be implied through a particular jargon or structure, rather than explicitly expressed in their content. Large language models (LLMs) hold great potential in identifying such implied relevance by leveraging their reasoning skills. Nevertheless, current LLM-augmented retrieval is hindered by high latency and computation cost, as the LLM typically computes the query-document relevance online, for every query anew. To tackle this issue we introduce EnrichIndex, a retrieval approach which instead uses the LLM offline to build semantically-enriched retrieval indices, by performing a single pass over all documents in the retrieval corpus once during ingestion time. Furthermore, the semantically-enriched indices can complement existing online retrieval approaches, boosting the performance of LLM re-rankers. We evaluated EnrichIndex on five retrieval tasks, involving passages and tables, and found that it outperforms strong online LLM-based retrieval systems, with an average improvement of 11.7 points in recall @ 10 and 10.6 points in NDCG @ 10 compared to strong baselines. In terms of online calls to the LLM, it processes 293.3 times fewer tokens which greatly reduces the online latency and cost. Overall, EnrichIndex is an effective way to build better retrieval indices offline by leveraging the strong reasoning skills of LLMs.

Paper Structure

This paper contains 29 sections, 1 equation, 3 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Enriching retrieval indices offline
  3. Object enrichment
  4. Summary.
  5. Purpose.
  6. QA pairs.
  7. Online retrieval
  8. Experiments
  9. Datasets
  10. Measuring retrieval performance and efficiency
  11. Baselines and setup
  12. Performance
  13. Performance of stage-one retrievers.
  14. Performance of stage-one retrievers with query expansion.
  15. Re-ranker performance.
  16. ...and 14 more sections

Figures (3)

  • Figure 1: EnrichIndex leverages LLMs offline to enrich each object, creating multiple semantically-enhanced indices. During online retrieval, it computes object relevance by calculating a weighted sum of similarities between the user query across all enriched indices: the original table, its summary, purpose and QA pairs. See Appendix \ref{['app:example']} for an additional example of enriching a free-form document, whose retrieval requires implicit reasoning.
  • Figure 2: Absolute improvement (in points) of retrieval performance with different types of enrichment relative to using only the original object content. Base refers to original object content, P refers to purpose, S refers to summary, and QA refers to QA pairs.
  • Figure 3: EnrichIndex leverages LLMs offline to enrich each object, creating multiple semantically-enhanced indices. During online retrieval, it computes object relevance by calculating a weighted sum of similarities between the user query across all enriched indices: the original document, its summary, purpose and QA pairs.