COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with Thought Processes

TL;DR

COIG-Writer tackles the scarcity of process-level supervision in Chinese creative writing by introducing a dataset of $1{,}665$ triplets across $51$ genres that pair reverse-engineered prompts, detailed thought processes, and final texts. The authors propose a two-component model where narrative logic, enhanced by process supervision, must be balanced with linguistic grounding drawn from general data; their experiments reveal a stabilization threshold around a $1{:}12$ creative-to-general data ratio and a language-specificity of creative capabilities, with strong gains in Chinese but limited cross-lingual transfer to English. They also introduce a comprehensive reverse-engineering pipeline and a 104-query evaluation benchmark to assess generation quality, coherence, and cultural appropriateness, highlighting that lexical diversity (TTR) can paradoxically correlate with lower quality. The work provides a framework for process-aware creative AI in data-scarce languages and underscores the need to treat narrative structure and surface language as complementary rather than interchangeable components. Overall, COIG-Writer demonstrates that explicit reasoning signals can substantially improve Chinese creative writing when paired with adequate general-language grounding, and it points to practical directions for scalable, cross-lingual evaluation of culturally grounded generation.

Abstract

Large language models exhibit systematic deficiencies in creative writing, particularly in non-English contexts where training data is scarce and lacks process-level supervision. We present COIG-Writer, a novel Chinese creative writing dataset that captures both diverse outputs and their underlying thought processes through systematic reverse-engineering of high-quality texts. Unlike existing datasets that provide only input-output pairs, COIG-Writer comprises 1,665 meticulously curated triplets spanning 51 genres, each containing: (1) a reverse-engineered prompt, (2) detailed creative reasoning documenting decision-making processes, and (3) the final text. Through comprehensive experiments, we identify a two-component model of creative writing: narrative logic (provided by process supervision) and linguistic expression (maintained by general-purpose data). Our findings reveal three critical insights: (1) Process supervision is highly effective but requires stabilization with general data. A ratio of at least one creative sample to twelve general samples is needed to achieve optimal performance; below this threshold, the win rate progressively degrades (from 62.75% down to 35.78%)., (2) creative capabilities are culturally-bound with no cross-lingual transfer (89.26pp gap between Chinese and English performance), and (3) lexical diversity inversely correlates with creative quality (TTR paradox), suggesting high diversity signals compensatory behavior for logical deficiencies. These findings establish that creative excellence emerges from the interaction between logical scaffolding and linguistic grounding, analogous to how mathematical reasoning enhances but cannot replace linguistic competence in foundation models.

Figures (8)

• Figure 1: Overview of COIG-Writer construction and evaluation.Stage 1 (Data Construction): High-quality Chinese texts spanning 51 genres are collected and filtered, followed by expert reverse-engineering to extract creative prompts and reasoning processes, yielding 1,665 validated triplets. Stage 2 (Human Evaluation): Models trained on COIG-Writer undergo rigorous human preference evaluation through pairwise comparisons, with analysis of win rates and lexical diversity (TTR) to assess creative writing quality.
• Figure 2: The data curation pipeline of COIG-Writer. Our methodology consists of three main stages: (1) Genre scope definition through expert consultation, (2) Multi-stage source text collection and filtering, and (3) Reverse-engineering of thought processes with comprehensive quality control.
• Figure 3: Dataset composition of COIG-Writer. (a) Distribution across 7 main categories encompassing 51 specific genres. Communication (28.9%) and Novel (28.0%) constitute the majority, followed by Non-fiction (14.6%) and Functional Writing (13.3%). (b) Length distributions for prompts (Query), reasoning processes (Thought), and articles (Answer) demonstrate the varying complexity across the dataset.
• Figure 4: Distribution of character counts across model variants. Box plots show median, IQR (box), whiskers (1.5×IQR), and outliers (dots). Both languages show $\mathcal{M}_{\text{CW}}$ producing shortest outputs, with Chinese texts generally shorter due to character density.
• Figure 5: The TTR Paradox. Higher lexical diversity correlates with lower creative quality. $\mathcal{M}_{\text{G}}$ achieves highest TTR (0.678) but loses to $\mathcal{M}_{\text{CW+10k}}$ (TTR = 0.593) with only 37.25% win rate, challenging conventional assumptions about diversity metrics.