Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generation
Pengyue Jia, Derong Xu, Xiaopeng Li, Zhaocheng Du, Xiangyang Li, Yichao Wang, Yuhao Wang, Qidong Liu, Maolin Wang, Huifeng Guo, Ruiming Tang, Xiangyu Zhao
TL;DR
This work tackles the misalignment between reranker relevance and generator reasoning in Retrieval-Augmented Generation by introducing RADIO, a rationale distillation framework. RADIO extracts explicit rationales with LLMs from the question-answer pair and uses these rationales to guide a rationale-based reranking, subsequently fine-tuning the reranker with InfoNCE to better support the generator. Through comprehensive experiments across open-domain QA, multi-choice, and multi-hop QA, RADIO consistently outperforms baselines and demonstrates robust transferability across different generators and rerankers. The approach offers a practical pathway to improve RAG systems by embedding explicit reasoning signals into the document selection process.
Abstract
The reranker and generator are two critical components in the Retrieval-Augmented Generation (i.e., RAG) pipeline, responsible for ranking relevant documents and generating responses. However, due to differences in pre-training data and objectives, there is an inevitable gap between the documents ranked as relevant by the reranker and those required by the generator to support answering the query. To address this gap, we propose RADIO, a novel and practical preference alignment framework with RAtionale DIstillatiOn. Specifically, we first propose a rationale extraction method that leverages the reasoning capabilities of Large Language Models (LLMs) to extract the rationales necessary for answering the query. Subsequently, a rationale-based alignment process is designed to rerank the documents based on the extracted rationales, and fine-tune the reranker to align the preferences. We conduct extensive experiments on two tasks across three datasets to demonstrate the effectiveness of our approach compared to baseline methods. Our code is released online to ease reproduction.