Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QChunker: Learning Question-Aware Text Chunking for Domain RAG via Multi-Agent Debate

Jihao Zhao, Daixuan Li, Pengfei Li, Shuaishuai Zu, Biao Qin, Hongyan Liu

Abstract

The effectiveness upper bound of retrieval-augmented generation (RAG) is fundamentally constrained by the semantic integrity and information granularity of text chunks in its knowledge base. To address these challenges, this paper proposes QChunker, which restructures the RAG paradigm from retrieval-augmentation to understanding-retrieval-augmentation. Firstly, QChunker models the text chunking as a composite task of text segmentation and knowledge completion to ensure the logical coherence and integrity of text chunks. Drawing inspiration from Hal Gregersen's "Questions Are the Answer" theory, we design a multi-agent debate framework comprising four specialized components: a question outline generator, text segmenter, integrity reviewer, and knowledge completer. This framework operates on the principle that questions serve as catalysts for profound insights. Through this pipeline, we successfully construct a high-quality dataset of 45K entries and transfer this capability to small language models. Additionally, to handle long evaluation chains and low efficiency in existing chunking evaluation methods, which overly rely on downstream QA tasks, we introduce a novel direct evaluation metric, ChunkScore. Both theoretical and experimental validations demonstrate that ChunkScore can directly and efficiently discriminate the quality of text chunks. Furthermore, during the text segmentation phase, we utilize document outlines for multi-path sampling to generate multiple candidate chunks and select the optimal solution employing ChunkScore. Extensive experimental results across four heterogeneous domains exhibit that QChunker effectively resolves aforementioned issues by providing RAG with more logically coherent and information-rich text chunks.

QChunker: Learning Question-Aware Text Chunking for Domain RAG via Multi-Agent Debate

Abstract

The effectiveness upper bound of retrieval-augmented generation (RAG) is fundamentally constrained by the semantic integrity and information granularity of text chunks in its knowledge base. To address these challenges, this paper proposes QChunker, which restructures the RAG paradigm from retrieval-augmentation to understanding-retrieval-augmentation. Firstly, QChunker models the text chunking as a composite task of text segmentation and knowledge completion to ensure the logical coherence and integrity of text chunks. Drawing inspiration from Hal Gregersen's "Questions Are the Answer" theory, we design a multi-agent debate framework comprising four specialized components: a question outline generator, text segmenter, integrity reviewer, and knowledge completer. This framework operates on the principle that questions serve as catalysts for profound insights. Through this pipeline, we successfully construct a high-quality dataset of 45K entries and transfer this capability to small language models. Additionally, to handle long evaluation chains and low efficiency in existing chunking evaluation methods, which overly rely on downstream QA tasks, we introduce a novel direct evaluation metric, ChunkScore. Both theoretical and experimental validations demonstrate that ChunkScore can directly and efficiently discriminate the quality of text chunks. Furthermore, during the text segmentation phase, we utilize document outlines for multi-path sampling to generate multiple candidate chunks and select the optimal solution employing ChunkScore. Extensive experimental results across four heterogeneous domains exhibit that QChunker effectively resolves aforementioned issues by providing RAG with more logically coherent and information-rich text chunks.
Paper Structure (33 sections, 2 theorems, 18 equations, 4 figures, 1 table)

This paper contains 33 sections, 2 theorems, 18 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Text Segmentation
  4. Text Chunking in RAG
  5. Text Chunking for Domain RAG
  6. Problem Definition
  7. Question-Aware Multi-Agent Debate
  8. Question Outline Generator
  9. Text Segmenter
  10. Integrity Reviewer
  11. Knowledge Completer
  12. Design and Validation of ChunkScore
  13. Logical Independence (LI)
  14. Semantic Dispersion (SD)
  15. ChunkScore
  16. ...and 18 more sections

Key Result

Lemma 1

For a set of $K$ vectors $\mathbf{V}=\{\mathbf{v}_1, \dots, \mathbf{v}_K\}$ where $\mathbf{v}_i \in \mathbb{R}^d$, the corresponding Gram matrix $\mathbf{G}$ is defined as $\mathbf{G}_{ij} = \mathbf{v}_i^\top \mathbf{v}_j$, i.e., $\mathbf{G}=\mathbf{V}^\top \mathbf{V}$. The determinant of this matri

Figures (4)

  • Figure 1: Overview of the QChunker framework's multi-agent debate process. The workflow comprises four key stages: question outline generation, text segmentation, integrity review, and knowledge completion, followed by a concrete example.
  • Figure 2: Correlation analysis between ChunkScore and ROUGE-L performance on the CRUD Benchmark.
  • Figure 3: Trends in perplexity variations between original and rewritten text chunks across different LLMs.
  • Figure 4: Output example of the QChunker framework when processing chemical documents.

Theorems & Definitions (2)

  • Lemma 1: Gram Matrix Determinant and Volume de2008use
  • Lemma 2: Log-Determinant and Differential Entropy cai2015law