Model Editing for New Document Integration in Generative Information Retrieval

TL;DR

This paper proposes DOME (docID-oriented model editing), a novel method that effectively and efficiently adapts GR models to unseen documents and achieves this with only about 60% of the training time required by incremental training, considerably reducing computational cost and enabling efficient, frequent model updates.

Abstract

Generative retrieval (GR) reformulates the Information Retrieval (IR) task as the generation of document identifiers (docIDs). Despite its promise, existing GR models exhibit poor generalization to newly added documents, often failing to generate the correct docIDs. While incremental training offers a straightforward remedy, it is computationally expensive, resource-intensive, and prone to catastrophic forgetting, thereby limiting the scalability and practicality of GR. In this paper, we identify the core bottleneck as the decoder's ability to map hidden states to the correct docIDs of newly added documents. Model editing, which enables targeted parameter modifications for docID mapping, represents a promising solution. However, applying model editing to current GR models is not trivial, which is severely hindered by indistinguishable edit vectors across queries, due to the high overlap of shared docIDs in retrieval results. To address this, we propose DOME (docID-oriented model editing), a novel method that effectively and efficiently adapts GR models to unseen documents. DOME comprises three stages: (1) identification of critical layers, (2) optimization of edit vectors, and (3) construction and application of updates. At its core, DOME employs a hybrid-label adaptive training strategy that learns discriminative edit vectors by combining soft labels, which preserve query-specific semantics for distinguishable updates, with hard labels that enforce precise mapping modifications. Experiments on widely used benchmarks, including NQ and MS MARCO, show that our method significantly improves retrieval performance on new documents while maintaining effectiveness on the original collection. Moreover, DOME achieves this with only about 60% of the training time required by incremental training, considerably reducing computational cost and enabling efficient, frequent model updates.

Figures (6)

• Figure 1: Behavioral analysis of the initial and new documents over the NQ dataset kwiatkowski2019natural using a hierarchical k-means-based docID GR model tay2022transformer. The model is trained on the initial document set, and docIDs for new documents are assigned using the same k-means clustering procedure. (a) T-SNE maaten2008visualizing visualization of intermediate representations, where different colors denote different docID prefixes (i.e., 10, 30, 50), with each prefix corresponding to a specific document type. Squares represent initial documents, and circles represent new documents. (b) Retrieval performance (Recall@1 and Recall@100) on the initial and new document sets.
• Figure 2: Percentage of pairwise cosine similarity of edit vector $\delta$ across GR edit requests applied to the GR method tay2022transformer on NQ kwiatkowski2019natural. Yellow denotes pairs with different target docID, blue denotes pairs with the same target docID.
• Figure 3: An overview of the proposed DOME framework. (a) A patching technique is used to diagnose the model and locate the critical layers responsible for docID mapping. (b) Our hybrid-label adaptive training strategy is employed to compute diverse and effective edit vectors ($\delta$) for the new mappings. For clarity, the diagram retains only the crucial $W_{\mathrm{out}}$ component from the FFN layer. (c) These edit vectors are then used to construct an update matrix ($\Delta$), which is applied to the crucial layer parameters ($W_{\text{out}}$) to inject the new docID mapping.
• Figure 4: Percentage of pairwise cosine similarity of edit vector $\delta$ across GR edit requests on NQ. Yellow denotes pairs with different target docID, blue denotes pairs with the same target docID.
• Figure 5: Accuracies of average patching across decoder layers for predicting docID tokens at positions 0–3 on NQ.