Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Collective Critics for Creative Story Generation

Minwook Bae, Hyounghun Kim

TL;DR

Long-form story generation with narrative coherence and reader engagement remains difficult for LLMs. CritiCS introduces a two-stage plan-to-text framework in which a team of LLM critics and a leader iteratively refine story plans (CrPlan) and prose (CrText) using explicit creative criteria, including persona-driven critiques. Extensive human evaluations show that CrPlan boosts plan creativity and coherence, while CrText enhances expressiveness in terms of image and voice without sacrificing coherence, across multiple backbones and baselines. The framework supports interactive human–machine collaboration and adaptable critique configurations, offering a scalable approach to more engaging AI-generated narratives.

Abstract

Generating a long story of several thousand words with narrative coherence using Large Language Models (LLMs) has been a challenging task. Previous research has addressed this challenge by proposing different frameworks that create a story plan and generate a long story based on that plan. However, these frameworks have been mainly focusing on maintaining narrative coherence in stories, often overlooking creativity in story planning and the expressiveness of the stories generated from those plans, which are desirable properties to captivate readers' interest. In this paper, we propose Collective Critics for Creative Story Generation framework (CritiCS), which is composed of plan refining stage (CrPlan) and story generation stage (CrText), to integrate a collective revision mechanism that promotes those properties into long-form story generation process. Specifically, in each stage, a group of LLM critics and one leader collaborate to incrementally refine drafts of plan and story throughout multiple rounds. Extensive human evaluation shows that the CritiCS can significantly enhance story creativity and reader engagement, while also maintaining narrative coherence. Furthermore, the design of the framework allows active participation from human writers in any role within the critique process, enabling interactive human-machine collaboration in story writing.

Collective Critics for Creative Story Generation

TL;DR

Long-form story generation with narrative coherence and reader engagement remains difficult for LLMs. CritiCS introduces a two-stage plan-to-text framework in which a team of LLM critics and a leader iteratively refine story plans (CrPlan) and prose (CrText) using explicit creative criteria, including persona-driven critiques. Extensive human evaluations show that CrPlan boosts plan creativity and coherence, while CrText enhances expressiveness in terms of image and voice without sacrificing coherence, across multiple backbones and baselines. The framework supports interactive human–machine collaboration and adaptable critique configurations, offering a scalable approach to more engaging AI-generated narratives.

Abstract

Generating a long story of several thousand words with narrative coherence using Large Language Models (LLMs) has been a challenging task. Previous research has addressed this challenge by proposing different frameworks that create a story plan and generate a long story based on that plan. However, these frameworks have been mainly focusing on maintaining narrative coherence in stories, often overlooking creativity in story planning and the expressiveness of the stories generated from those plans, which are desirable properties to captivate readers' interest. In this paper, we propose Collective Critics for Creative Story Generation framework (CritiCS), which is composed of plan refining stage (CrPlan) and story generation stage (CrText), to integrate a collective revision mechanism that promotes those properties into long-form story generation process. Specifically, in each stage, a group of LLM critics and one leader collaborate to incrementally refine drafts of plan and story throughout multiple rounds. Extensive human evaluation shows that the CritiCS can significantly enhance story creativity and reader engagement, while also maintaining narrative coherence. Furthermore, the design of the framework allows active participation from human writers in any role within the critique process, enabling interactive human-machine collaboration in story writing.
Paper Structure (45 sections, 5 figures, 42 tables, 2 algorithms)

This paper contains 45 sections, 5 figures, 42 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Long-form Story Generation.
  4. Creative Writing.
  5. Collaborative Writing.
  6. Interactive Agents.
  7. Collective Critics for Creative Story Generation
  8. CrPlan
  9. CrText
  10. Experiments
  11. CrPlan Evaluation
  12. Generation Setting.
  13. CrText Evaluation
  14. Generation Setting.
  15. Analysis
  16. ...and 30 more sections

Figures (5)

  • Figure 1: The framework comprises two stages: CrPlan and CrText. CrPlan involves five phases: creating a story plan from a premise, reviewing the plan with critics' persona-driven perspectives, selecting a critique for revision by a leader, storing the revised plan, and choosing a plan for further development by an evaluator. Personas of critics are created based on the themes or content of the narratives, which helps in generating detailed and contextually relevant critiques. Please refer to Appendix \ref{['abl:persona_adpative']} critiques and refines long story text based on creative criteria, with a leader selecting and improving the best expressions.
  • Figure 2: Pairwise story plan comparisons using GPT-4: Non-Persona-Critics vs. Persona-Critics.
  • Figure 3: Pairwise comparison evaluations using GPT-4 (Win Rate %) to analyze changes in story plan creativity and coherence with different critique iterations.
  • Figure 4: Machine-human interactive writing system allows human participation as any of the players in the revision process of CritiCS.
  • Figure 5: Web application for human-machine interactive writing.