Incorporating Q&A Nuggets into Retrieval-Augmented Generation

TL;DR

The paper tackles citation grounding in Retrieval-Augmented Generation by introducing Crucible, a nugget-first RAG that builds a bank of Q&A nuggets from retrieved documents and uses them to guide retrieval, extraction, and final assembly while preserving explicit provenance. It formalizes nugget quality metrics and a ranking pipeline, and demonstrates substantial gains over a recent nugget-based system on the NeuCLIR 2024 benchmark, including improvements in Nugget Recall and Nugget Density, using a per-nugget extraction flow with optional verification. Crucible represents a practical realization of RAGE (retrieval-augmented generation with automatic evaluation), showing how explicit, testable nugget units can improve factual grounding and citation traceability in long-form generation. The study discusses computational costs, potential limitations, and avenues for broader adoption and future Auto-Judge evaluation tracks.

Abstract

RAGE systems integrate ideas from automatic evaluation (E) into Retrieval-augmented Generation (RAG). As one such example, we present Crucible, a Nugget-Augmented Generation System that preserves explicit citation provenance by constructing a bank of Q&A nuggets from retrieved documents and uses them to guide extraction, selection, and report generation. Reasoning on nuggets avoids repeated information through clear and interpretable Q&A semantics - instead of opaque cluster abstractions - while maintaining citation provenance throughout the entire generation process. Evaluated on the TREC NeuCLIR 2024 collection, our Crucible system substantially outperforms Ginger, a recent nugget-based RAG system, in nugget recall, density, and citation grounding.

Figures (2)

• Figure 1: For each generated nugget, Crucible extracts candidate sentences and adds the best $k$ sentences to the final response. No content clustering is needed.
• Figure 2: Prompt for scanning, extraction, and sentence candidate generation.