Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI4Reading: Chinese Audiobook Interpretation System Based on Multi-Agent Collaboration

Minjiang Huang, Jipeng Qiang, Yi Zhu, Chaowei Zhang, Xiangyu Zhao, Kui Yu

TL;DR

AI4Reading presents a multi-agent framework that leverages large language models and speech synthesis to automatically generate interpretative audiobooks in Chinese. The system decomposes the task into 11 specialized agents spanning topic analysis, case enrichment, editing, rewriting, and proofreading, followed by audio production with Fish-Speech TTS. Experiments against FanDeng show the manuscripts are simpler and more accurate, while the synthesized speech is slightly less natural, indicating a strength in content quality and a remaining gap in speech realism. The work demonstrates a scalable, language-accessible approach to interpretative publishing, with ethical considerations and clear avenues for expanding genre coverage and cross-language interpretation.

Abstract

Audiobook interpretations are attracting increasing attention, as they provide accessible and in-depth analyses of books that offer readers practical insights and intellectual inspiration. However, their manual creation process remains time-consuming and resource-intensive. To address this challenge, we propose AI4Reading, a multi-agent collaboration system leveraging large language models (LLMs) and speech synthesis technology to generate podcast, like audiobook interpretations. The system is designed to meet three key objectives: accurate content preservation, enhanced comprehensibility, and a logical narrative structure. To achieve these goals, we develop a framework composed of 11 specialized agents,including topic analysts, case analysts, editors, a narrator, and proofreaders that work in concert to explore themes, extract real world cases, refine content organization, and synthesize natural spoken language. By comparing expert interpretations with our system's output, the results show that although AI4Reading still has a gap in speech generation quality, the generated interpretative scripts are simpler and more accurate.

AI4Reading: Chinese Audiobook Interpretation System Based on Multi-Agent Collaboration

TL;DR

AI4Reading presents a multi-agent framework that leverages large language models and speech synthesis to automatically generate interpretative audiobooks in Chinese. The system decomposes the task into 11 specialized agents spanning topic analysis, case enrichment, editing, rewriting, and proofreading, followed by audio production with Fish-Speech TTS. Experiments against FanDeng show the manuscripts are simpler and more accurate, while the synthesized speech is slightly less natural, indicating a strength in content quality and a remaining gap in speech realism. The work demonstrates a scalable, language-accessible approach to interpretative publishing, with ethical considerations and clear avenues for expanding genre coverage and cross-language interpretation.

Abstract

Audiobook interpretations are attracting increasing attention, as they provide accessible and in-depth analyses of books that offer readers practical insights and intellectual inspiration. However, their manual creation process remains time-consuming and resource-intensive. To address this challenge, we propose AI4Reading, a multi-agent collaboration system leveraging large language models (LLMs) and speech synthesis technology to generate podcast, like audiobook interpretations. The system is designed to meet three key objectives: accurate content preservation, enhanced comprehensibility, and a logical narrative structure. To achieve these goals, we develop a framework composed of 11 specialized agents,including topic analysts, case analysts, editors, a narrator, and proofreaders that work in concert to explore themes, extract real world cases, refine content organization, and synthesize natural spoken language. By comparing expert interpretations with our system's output, the results show that although AI4Reading still has a gap in speech generation quality, the generated interpretative scripts are simpler and more accurate.
Paper Structure (22 sections, 2 equations, 6 figures, 1 table)

This paper contains 22 sections, 2 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Audiobook System
  4. Interpretation Generation
  5. System Design
  6. Interpretation Script Generation
  7. Topics & Cases Identification (TCI)
  8. Preliminary Interpretation (PI)
  9. Oral Rewriting (OR)
  10. Reconstruction and Revision (RR)
  11. Audio Generation
  12. Agent Configuration Rationale
  13. Experiments and Evaluation
  14. Experimental Setup
  15. Evaluation Metrics
  16. ...and 7 more sections

Figures (6)

  • Figure 1: Flowchart of expert-based and LLM-based audiobook interpretation system.
  • Figure 2: The process of interpretation script generation in AI4Reading based on multi-agent collaboration.
  • Figure 3: The time spent by the 7 evaluators on each element of the evaluation, with invalid 1 and invalid 2 being the users who discarded the results.
  • Figure 4: Screenshot of audio evaluation.
  • Figure 5: Screenshot of interpretation script evaluation.
  • ...and 1 more figures