RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery

TL;DR

RaPID tackles knowledge-intensive long-text generation by addressing hallucinations, coherence, and efficiency in wiki-style article creation. It introduces three core modules: retrieval-augmented outline generation, attribute-constrained information discovery, and plan-guided article generation. Experiments on FreshWiki-2024 show RaPID outperforms baselines across outline quality, article quality, factuality, and information diversity while reducing latency. A robust writing plan in the form of a topological DAG helps maintain coherence across long documents. The work provides a scalable framework and a substantial dataset to advance retrieval-augmented long-form writing.

Abstract

Generating knowledge-intensive and comprehensive long texts, such as encyclopedia articles, remains significant challenges for Large Language Models. It requires not only the precise integration of facts but also the maintenance of thematic coherence throughout the article. Existing methods, such as direct generation and multi-agent discussion, often struggle with issues like hallucinations, topic incoherence, and significant latency. To address these challenges, we propose RAPID, an efficient retrieval-augmented long text generation framework. RAPID consists of three main modules: (1) Retrieval-augmented preliminary outline generation to reduce hallucinations, (2) Attribute-constrained search for efficient information discovery, (3) Plan-guided article generation for enhanced coherence. Extensive experiments on our newly compiled benchmark dataset, FreshWiki-2024, demonstrate that RAPID significantly outperforms state-of-the-art methods across a wide range of evaluation metrics (e.g. long-text generation, outline quality, latency, etc). Our work provides a robust and efficient solution to the challenges of automated long-text generation.

Figures (17)

• Figure 1: An example of generating a wiki-style article using various methods highlights distinct challenges and considerations. Direct generation may suffer from the large model's limited internal knowledge. While methods based on multi-agent discussions can provide broad coverage of the topic, they may also result in increased hallucinations and reduced efficiency.
• Figure 2: The framework of RaPID, which consists of three main stages: (a) Retrieval-Augmented Outline Generation, where an initial outline is created based on a brief introduction and examples; (b) Attribute-Constrained Search, which leverages an attribute-based mechanism to discover relevant information and refine the outline accordingly; and (c) Plan-Guided Article Generation, where a structured writing plan is developed based on dependencies between sections, resulting in a more coherent and fluent article. The blue dashed lines illustrate how the outline evolves throughout the processes of information discovery and writing planning.
• Figure 3: The distribution of the length of FreshWiki-2024
• Figure 4: The distribution of classification of Freshwiki-2024
• Figure 5: The distribution of time consumed in each stage of the pipelines.