Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

Lichang Song, Ting Long, Yi Chang

TL;DR

This work reformulate RAG as a cooperative multi-agent decision-making problem and proposes Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-makers rather than being connected through an asymmetric dependency pipeline.

Abstract

Retrieval-Augmented Generation (RAG) has demonstrated strong effectiveness in knowledge-intensive tasks by grounding language generation in external evidence. Despite its success, many existing RAG systems are built based on a ranking-centric, asymmetric dependency paradigm, where the generation quality of the generator is highly dependent on reranking results of the reranker. To overcome this limitation, we reformulate RAG as a cooperative multi-agent decision-making problem and propose Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-makers rather than being connected through an asymmetric dependency pipeline. By jointly optimizing their behaviors toward a shared task objective, the reranker and generator are encouraged to cooperate, ensuring that document reranking and generation work in concert to improve the final response. Experimental results demonstrate good generalization and improved generation stability of CoRAG, even when the model is trained on only around 10K PopQA samples. Our model released in https://anonymous.4open.science/r/CoRAG-D63F

Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

TL;DR

This work reformulate RAG as a cooperative multi-agent decision-making problem and proposes Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-makers rather than being connected through an asymmetric dependency pipeline.

Abstract

Retrieval-Augmented Generation (RAG) has demonstrated strong effectiveness in knowledge-intensive tasks by grounding language generation in external evidence. Despite its success, many existing RAG systems are built based on a ranking-centric, asymmetric dependency paradigm, where the generation quality of the generator is highly dependent on reranking results of the reranker. To overcome this limitation, we reformulate RAG as a cooperative multi-agent decision-making problem and propose Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-makers rather than being connected through an asymmetric dependency pipeline. By jointly optimizing their behaviors toward a shared task objective, the reranker and generator are encouraged to cooperate, ensuring that document reranking and generation work in concert to improve the final response. Experimental results demonstrate good generalization and improved generation stability of CoRAG, even when the model is trained on only around 10K PopQA samples. Our model released in https://anonymous.4open.science/r/CoRAG-D63F
Paper Structure (24 sections, 11 equations, 4 figures, 7 tables, 1 algorithm)

This paper contains 24 sections, 11 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Method
  4. The Reranker
  5. The Generator
  6. The Optimization
  7. Reranker Optimization.
  8. Generator Optimization.
  9. Experiments
  10. Experimental Setting
  11. Datasets
  12. Evaluation Metrics.
  13. Baselines.
  14. Implementation Details
  15. Main Results (RQ1)
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Comparison with previous works. Previous works assume an asymmetric dependency between the reranker and the generator, whereas CoRAG models them as cooperative agents in a multi-agent reinforcement learning framework.
  • Figure 2: CoRAG overview. The reranker and generator cooperate to generate responses. The task-oriented reward derived from response guides GRPO-aligned training of the reranker and GRPO optimization of the generator.
  • Figure 3: Cross-validation results with different reranker and generator combinations (Top-3 setting).
  • Figure 4: Impact of the document number.