The Role of Parametric Injection-A Systematic Study of Parametric Retrieval-Augmented Generation

TL;DR

This work systematically analyzes parametric retrieval-augmented generation (PRAG), where documents are encoded as model parameters (e.g., LoRA) and injected during inference. It finds that such parametric representations store partial, often high-level semantic knowledge, enabling better context understanding and robustness to noisy retrieval, but they do not supplant standard text-based retrieval, which preserves fine-grained facts. A hybrid approach (PRAG-Combine) that blends parametric knowledge with textual documents yields the strongest overall performance, albeit at additional storage and computation costs. The study highlights a key direction for PRAG: enriching parametric representations to capture richer factual content to close the gap with conventional RAG and realize practical efficiency gains.

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by retrieving external documents. As an emerging form of RAG, parametric retrieval-augmented generation (PRAG) encodes documents as model parameters (i.e., LoRA modules) and injects these representations into the model during inference, enabling interaction between the LLM and documents at parametric level. Compared with directly placing documents in the input context, PRAG is more efficient and has the potential to offer deeper model-document interaction. Despite its growing attention, the mechanism underlying parametric injection remains poorly understood. In this work, we present a systematic study of PRAG to clarify the role of parametric injection, showing that parameterized documents capture only partial semantic information of documents, and relying on them alone yields inferior performance compared to interaction at text level. However, these parametric representations encode high-level document information that can enhance the model's understanding of documents within the input context. When combined parameterized documents with textual documents, the model can leverage relevant information more effectively and become more robust to noisy inputs, achieving better performance than either source alone. We recommend jointly using parameterized and textual documents and advocate for increasing the information content of parametric representations to advance PRAG.

Figures (12)

• Figure 1: Performance of different methods on the new-knowledge dataset, where the model must rely on external documents to answer questions correctly. LLaMA-1B denotes LLaMA3.2-1B-Instruct, Qwen-1.5 denotes Qwen2.5-1.5B-Instruct, and Qwen-7B denotes Qwen2.5-7B-Instruct.
• Figure 2: An example from the new-knowledge dataset. When asked a question about an event outside its training data, the Vanilla LLM hallucinates, whereas PRAG produces the correct answer. More cases are provided in Appendix \ref{['app:more_examples']}.
• Figure 3: Similarity distribution of LoRA modules (averaged over all layers) for Qwen2.5-1.5B-Instruct. Relevant passage pairs exhibit slightly higher similarity than irrelevant pairs, implying that LoRAs encode some semantic and factual content. However, the gap is modest, suggesting limited encoding of fine-grained, document-specific knowledge.
• Figure 4: Difference in parametric knowledge score between LLMs with and without parametric injection. Most comparisons show a positive average difference, and all models exhibit a pronounced increase in the final layers, indicating that parametric injection increases parametric knowledge in the residual stream and that the representations encode high-level knowledge.
• Figure 5: Performance of different methods on fact-checking and slot filling. LLaMA-1B denotes LLaMA3.2-1B-Instruct, Qwen-1.5 denotes Qwen2.5-1.5B-Instruct, and Qwen-7B denotes Qwen2.5-7B-Instruct. Methods with parametric injection achieve stronger performance on both tasks, indicating that the benefits generalize to other downstream tasks.