Fine-Grained Guidance for Retrievers: Leveraging LLMs' Feedback in Retrieval-Augmented Generation

TL;DR

This work proposes a novel framework, FiGRet (Fine-grained Guidance for Retrievers), which leverages the language capabilities of LLMs to construct examples from a more granular, information-centric perspective to guide the learning of retrievers.

Abstract

Retrieval-Augmented Generation (RAG) has proven to be an effective method for mitigating hallucination issues inherent in large language models (LLMs). Previous approaches typically train retrievers based on semantic similarity, lacking optimization for RAG. More recent works have proposed aligning retrievers with the preference signals of LLMs. However, these preference signals are often difficult for dense retrievers, which typically have weaker language capabilities, to understand and learn effectively. Drawing inspiration from pedagogical theories like Guided Discovery Learning, we propose a novel framework, FiGRet (Fine-grained Guidance for Retrievers), which leverages the language capabilities of LLMs to construct examples from a more granular, information-centric perspective to guide the learning of retrievers. Specifically, our method utilizes LLMs to construct easy-to-understand examples from samples where the retriever performs poorly, focusing on three learning objectives highly relevant to the RAG scenario: relevance, comprehensiveness, and purity. These examples serve as scaffolding to ultimately align the retriever with the LLM's preferences. Furthermore, we employ a dual curriculum learning strategy and leverage the reciprocal feedback between LLM and retriever to further enhance the performance of the RAG system. A series of experiments demonstrate that our proposed framework enhances the performance of RAG systems equipped with different retrievers and is applicable to various LLMs.

Figures (4)

• Figure 1: An illustration of the guided construction process within the FiGRet framework. The colored blocks represent the perspective of treating documents as collections of information units from an information-centric perspective. The top-left corner depicts the retrieval and training process of a typical RAG system. The bottom-left corner illustrates the previous approach, which considers each document as a monolithic unit, leading to bias due to the erroneous convergence or divergence of certain information units mistakenly deemed relevant or irrelevant (indicated by the red question marks). Our approach, depicted on the right, mitigates this bias by constructing guided examples through an information-centric perspective, facilitating targeted learning towards specific objectives. In this depiction, "Pos." represents Positive, "Neg." represents Negative, and "Comp." represents Comprehensive.
• Figure 2: Objective-wise Retrieval Improvements After Training. The retriever demonstrates improved performance across all three objectives
• Figure 3: Aspect-wise Retrieval Improvements After Training.
• Figure 4: Showcase of Guidance Example Construction.