Generative Dense Retrieval: Memory Can Be a Burden

TL;DR

The paper addresses the limitations of Generative Retrieval (GR) in large corpora by proposing Generative Dense Retrieval (GDR), a coarse-to-fine paradigm that combines memorizing-based inter-cluster mapping with memorizing-free intra-cluster dense matching. GDR uses memory-friendly cluster identifiers to recall relevant document clusters and dense representations for fine-grained intra-cluster retrieval, augmented by cluster-adaptive negative sampling. Empirical results on Natural Questions show that GDR achieves superior recall (R@k) and robust scalability compared to sparse, dense, and generative baselines, while maintaining competitive efficiency. The approach offers a practical path to scalable, high-quality retrieval in large-scale ecosystems where corpus updates and expansions are frequent.

Abstract

Generative Retrieval (GR), autoregressively decoding relevant document identifiers given a query, has been shown to perform well under the setting of small-scale corpora. By memorizing the document corpus with model parameters, GR implicitly achieves deep interaction between query and document. However, such a memorizing mechanism faces three drawbacks: (1) Poor memory accuracy for fine-grained features of documents; (2) Memory confusion gets worse as the corpus size increases; (3) Huge memory update costs for new documents. To alleviate these problems, we propose the Generative Dense Retrieval (GDR) paradigm. Specifically, GDR first uses the limited memory volume to achieve inter-cluster matching from query to relevant document clusters. Memorizing-free matching mechanism from Dense Retrieval (DR) is then introduced to conduct fine-grained intra-cluster matching from clusters to relevant documents. The coarse-to-fine process maximizes the advantages of GR's deep interaction and DR's scalability. Besides, we design a cluster identifier constructing strategy to facilitate corpus memory and a cluster-adaptive negative sampling strategy to enhance the intra-cluster mapping ability. Empirical results show that GDR obtains an average of 3.0 R@100 improvement on NQ dataset under multiple settings and has better scalability.

Figures (2)

• Figure 1: Illustration of Dense Retrieval, Generative Retrieval and Generative Dense Retrieval.
• Figure 2: $\mathrm{R}@100$ descent rate of different types of methods when scaling to larger corpus.