Optimizing Knowledge Integration in Retrieval-Augmented Generation with Self-Selection

TL;DR

This work tackles the challenge of integrating internal parametric knowledge with external retrieved knowledge in retrieval-augmented generation (RAG) by introducing Self-Selection RAG, where an LLM selects between an internal-answer and an external-answer. To strengthen selection and generation, it adds Self-Selection-RGP, training LLMs with Direct Preference Optimization on a curated Retrieval-Generation Preference dataset. Experiments on Natural Questions and TriviaQA using Mistral-7B and Llama2-13B-Chat show that Self-Selection-RGP yields robust gains across retrieval settings and tasks, and ablation analyses corroborate the importance of data augmentation and preference alignment. The approach advances robust, less-noisy knowledge integration in LLMs, with practical impact for improving accuracy and reliability in RAG-driven QA systems.

Abstract

Retrieval-Augmented Generation (RAG), which integrates external knowledge into Large Language Models (LLMs), has proven effective in enabling LLMs to produce more accurate and reliable responses. However, it remains a significant challenge how to effectively integrate external retrieved knowledge with internal parametric knowledge in LLMs. In this work, we propose a novel Self-Selection RAG framework, where the LLM is made to select from pairwise responses generated with internal parametric knowledge solely and with external retrieved knowledge together to achieve enhanced accuracy. To this end, we devise a Self-Selection-RGP method to enhance the capabilities of the LLM in both generating and selecting the correct answer, by training the LLM with Direct Preference Optimization (DPO) over a curated Retrieval Generation Preference (RGP) dataset. Experimental results with two open-source LLMs (i.e., Llama2-13B-Chat and Mistral-7B) well demonstrate the superiority of our approach over other baseline methods on Natural Questions (NQ) and TrivialQA datasets.

Figures (6)

• Figure 1: An illustration of the proposed Self-Selection framework. For a given query, an LLM is first requested to generate the answers and their respective explanations with and without external knowledge. Then, the LLM is adopted to take them as input and choose one from them as the final answer with its explanation.
• Figure 2: An illustration of the proposed Self-Selection-RGP method.
• Figure 3: An illustration of the effects of using a different retriever. Our Self-Selection-RGP and Standard RAG both use BM25 as the new retriever. We adopt Mistral-7B as the base LLM for Standard RAG (7B) and Self-Selection-RGP (7B), and Llama2-13B-Chat for Standard RAG (13B) and Self-Selection-RGP (13B).
• Figure 4: An illustration of the effects of varying the number of retrieved passages.
• Figure 5: Ablation study. "Std RAG" refers to Standard RAG; "w/o Aug" indicates the method without Dataset Augmentation; "w/o Align" denotes the method without Preference Alignment; and "SS-RGP" represents our proposed Self-Selection RAG method.