Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CoCo Matrix: Taxonomy of Cognitive Contributions in Co-writing with Intelligent Agents

Ruyuan Wan, Simret Gebreegziabhe, Toby Jia-Jun Li, Karla Badillo-Urquiola

TL;DR

CoCo Matrix addresses the gap in evaluating human–AI co-writing by deploying entropy and information gain to map cognitive contributions across the writing process. It adapts Flower and Hayes' cognitive process theory to a two-dimensional taxonomy and applies it to 34 ACM systems, revealing a predominance of high entropy and high information gain collaborations and highlighting under-explored low-entropy, high-information-gain opportunities. The framework enables assessment of the writer's mental model through minimal text changes and supports process-aware evaluation and tool design. These insights guide the development of intelligent writing assistants that augment human creativity while preserving writer agency.

Abstract

In recent years, there has been a growing interest in employing intelligent agents in writing. Previous work emphasizes the evaluation of the quality of end product-whether it was coherent and polished, overlooking the journey that led to the product, which is an invaluable dimension of the creative process. To understand how to recognize human efforts in co-writing with intelligent writing systems, we adapt Flower and Hayes' cognitive process theory of writing and propose CoCo Matrix, a two-dimensional taxonomy of entropy and information gain, to depict the new human-agent co-writing model. We define four quadrants and situate thirty-four published systems within the taxonomy. Our research found that low entropy and high information gain systems are under-explored, yet offer promising future directions in writing tasks that benefit from the agent's divergent planning and the human's focused translation. CoCo Matrix, not only categorizes different writing systems but also deepens our understanding of the cognitive processes in human-agent co-writing. By analyzing minimal changes in the writing process, CoCo Matrix serves as a proxy for the writer's mental model, allowing writers to reflect on their contributions. This reflection is facilitated through the measured metrics of information gain and entropy, which provide insights irrespective of the writing system used.

CoCo Matrix: Taxonomy of Cognitive Contributions in Co-writing with Intelligent Agents

TL;DR

CoCo Matrix addresses the gap in evaluating human–AI co-writing by deploying entropy and information gain to map cognitive contributions across the writing process. It adapts Flower and Hayes' cognitive process theory to a two-dimensional taxonomy and applies it to 34 ACM systems, revealing a predominance of high entropy and high information gain collaborations and highlighting under-explored low-entropy, high-information-gain opportunities. The framework enables assessment of the writer's mental model through minimal text changes and supports process-aware evaluation and tool design. These insights guide the development of intelligent writing assistants that augment human creativity while preserving writer agency.

Abstract

In recent years, there has been a growing interest in employing intelligent agents in writing. Previous work emphasizes the evaluation of the quality of end product-whether it was coherent and polished, overlooking the journey that led to the product, which is an invaluable dimension of the creative process. To understand how to recognize human efforts in co-writing with intelligent writing systems, we adapt Flower and Hayes' cognitive process theory of writing and propose CoCo Matrix, a two-dimensional taxonomy of entropy and information gain, to depict the new human-agent co-writing model. We define four quadrants and situate thirty-four published systems within the taxonomy. Our research found that low entropy and high information gain systems are under-explored, yet offer promising future directions in writing tasks that benefit from the agent's divergent planning and the human's focused translation. CoCo Matrix, not only categorizes different writing systems but also deepens our understanding of the cognitive processes in human-agent co-writing. By analyzing minimal changes in the writing process, CoCo Matrix serves as a proxy for the writer's mental model, allowing writers to reflect on their contributions. This reflection is facilitated through the measured metrics of information gain and entropy, which provide insights irrespective of the writing system used.
Paper Structure (15 sections, 2 equations, 2 figures, 1 table)

This paper contains 15 sections, 2 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Cognitive Process Theory of Writing
  4. Intelligent Writing Assistant Tools
  5. Evaluation of Creative Natural Language Generation Systems
  6. Methodology
  7. Literature Review
  8. Metrics for Writing Process
  9. Construction of a Two-Dimensional Taxonomy
  10. CoCo Matrix: Taxonomy to Assess Cognitive Contributions
  11. High entropy, High information gain
  12. High entropy, Low information gain
  13. Low entropy, High information gain
  14. Low entropy, Low information gain
  15. Discussion

Figures (2)

  • Figure 1: Adapted Cognitive Process Theory of Writing: The cyan color frames are the original structure of the writing model from Flower et al.flower1981cognitive, we propose use entropy and information gain to depict the new model of human-agent co-writing.
  • Figure 2: CoCo Matrix - a two-dimensional matrix of entropy and information gain: we analyzed thirty-four systems with our taxonomy. Low entropy and high information gain systems are under-explored.