Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAG: Style-Aligned Article Generation via Model Collaboration

Chenning Xu, Fangxun Shu, Dian Jin, Jinghao Wei, Hao Jiang

TL;DR

The paper addresses the challenge of generating stylized articles that faithfully convey content. It introduces SAG, a collaborative framework that freezes a large language model to retain instruction-following and trains a smaller language model with style data, guided by a self-improvement style filter and a DPO-based hallucination mitigation stage. A new NoteBench benchmark is proposed to evaluate style imitation and faithfulness. Experiments show state-of-the-art gains in ROUGE-L and BLEU-4 over GPT-4 and vanilla SFT, while substantially reducing hallucinations, highlighting practical potential for controllable, style-aligned AI content generation.

Abstract

Large language models (LLMs) have increased the demand for personalized and stylish content generation. However, closed-source models like GPT-4 present limitations in optimization opportunities, while the substantial training costs and inflexibility of open-source alternatives, such as Qwen-72B, pose considerable challenges. Conversely, small language models (SLMs) struggle with understanding complex instructions and transferring learned capabilities to new contexts, often exhibiting more pronounced limitations. In this paper, we present a novel collaborative training framework that leverages the strengths of both LLMs and SLMs for style article generation, surpassing the performance of either model alone. We freeze the LLMs to harness their robust instruction-following capabilities and subsequently apply supervised fine-tuning on the SLM using style-specific data. Additionally, we introduce a self-improvement method to enhance style consistency. Our new benchmark, NoteBench, thoroughly evaluates style-aligned generation. Extensive experiments show that our approach achieves state-of-the-art performance, with improvements of 0.78 in ROUGE-L and 0.55 in BLEU-4 scores compared to GPT-4, while maintaining a low hallucination rate regarding factual and faithfulness.

SAG: Style-Aligned Article Generation via Model Collaboration

TL;DR

The paper addresses the challenge of generating stylized articles that faithfully convey content. It introduces SAG, a collaborative framework that freezes a large language model to retain instruction-following and trains a smaller language model with style data, guided by a self-improvement style filter and a DPO-based hallucination mitigation stage. A new NoteBench benchmark is proposed to evaluate style imitation and faithfulness. Experiments show state-of-the-art gains in ROUGE-L and BLEU-4 over GPT-4 and vanilla SFT, while substantially reducing hallucinations, highlighting practical potential for controllable, style-aligned AI content generation.

Abstract

Large language models (LLMs) have increased the demand for personalized and stylish content generation. However, closed-source models like GPT-4 present limitations in optimization opportunities, while the substantial training costs and inflexibility of open-source alternatives, such as Qwen-72B, pose considerable challenges. Conversely, small language models (SLMs) struggle with understanding complex instructions and transferring learned capabilities to new contexts, often exhibiting more pronounced limitations. In this paper, we present a novel collaborative training framework that leverages the strengths of both LLMs and SLMs for style article generation, surpassing the performance of either model alone. We freeze the LLMs to harness their robust instruction-following capabilities and subsequently apply supervised fine-tuning on the SLM using style-specific data. Additionally, we introduce a self-improvement method to enhance style consistency. Our new benchmark, NoteBench, thoroughly evaluates style-aligned generation. Extensive experiments show that our approach achieves state-of-the-art performance, with improvements of 0.78 in ROUGE-L and 0.55 in BLEU-4 scores compared to GPT-4, while maintaining a low hallucination rate regarding factual and faithfulness.
Paper Structure (22 sections, 3 equations, 1 figure, 8 tables, 1 algorithm)

This paper contains 22 sections, 3 equations, 1 figure, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Style-Aligned Article Generation
  4. Task Definition
  5. Style Data Curation
  6. Data Curation
  7. Self-improvement Style Filter
  8. Collaborative Training
  9. Simulating Intention with LLM.
  10. Imitating Style with SLM.
  11. Mitigating Hallucination with DPO.
  12. Experiment
  13. Experimental Settings
  14. Main Results
  15. Style Consistency.
  16. ...and 7 more sections

Figures (1)

  • Figure 1: The framework of the collaborative training of style-aligned article generation: User instruction is initially processed by a pre-trained LLM such as GPT4 to formulate neutral content. Subsequently, the style reference content, user instruction, and neutral content are fed to an SLM to generate stylish content.