Navigating the Path of Writing: Outline-guided Text Generation with Large Language Models

TL;DR

This paper introduces WritingPath, an outline-guided, five-step framework that uses explicit metadata, a structured outline, information browsing, and outline augmentation to steer large language models toward high-quality, goal-oriented writing. By integrating a hybrid evaluation framework—automatic outline and writing metrics plus human assessments—the authors demonstrate that augmented outlines significantly improve coherence, reduce redundancy, and better reflect user intent across multiple LLMs. The approach is validated on a diverse Korean blog dataset and deployed in a real-world NAVER platform, showing practical viability for AI-assisted writing in production. Key contributions include the design of a comprehensive proof-of-concept pipeline, a multi-faceted evaluation scheme, and evidence that outline augmentation yields tangible quality gains in free-form writing.

Abstract

Large Language Models (LLMs) have impacted the writing process, enhancing productivity by collaborating with humans in content creation platforms. However, generating high-quality, user-aligned text to satisfy real-world content creation needs remains challenging. We propose WritingPath, a framework that uses explicit outlines to guide LLMs in generating goal-oriented, high-quality text. Our approach draws inspiration from structured writing planning and reasoning paths, focusing on reflecting user intentions throughout the writing process. To validate our approach in real-world scenarios, we construct a diverse dataset from unstructured blog posts to benchmark writing performance and introduce a comprehensive evaluation framework assessing the quality of outlines and generated texts. Our evaluations with various LLMs demonstrate that the WritingPath approach significantly enhances text quality according to evaluations by both LLMs and professional writers.

Figures (13)

• Figure 1: Comparative overview of writing approaches: (A) direct generation, (B) iterative writing involving planning, editing, and explaining, and (C) WritingPath method, which starts with a consistency-focused plan, incorporates information-rich browsing, and results in an augmented, consistent, and rich outline.
• Figure 2: Main architecture of WritingPath, our proposed framework for guiding LLMs to generate high-quality text following a structured writing process. The WritingPath condenses text generation into five key steps. Inspired by human writing planning, it ensures alignment with specified writing goals.
• Figure 3: Breakdown of the seven key aspects used in writing evaluation, each with corresponding sub-aspects, employed in a Boolean QA-style checklist for human and LLM evaluation. This comprehensive framework ensures a multi-dimensional analysis of text quality.
• Figure 4: Overview of the main analysis steps in the WritingPath framework, covering meta-data only, initial outline, and augmented outline scenarios, respectively.
• Figure 5: Main analysis steps on writing evaluation results by (a) LLM and (b) Human Evaluation.