More Human or More AI? Visualizing Human-AI Collaboration Disclosures in Journalistic News Production

TL;DR

This study tackles the challenge of transparently communicating human–AI collaboration in journalism by designing and evaluating four visualization prototypes (Textual Disclosure, Role-based Timeline, Chatbot, Task-based Timeline). Through co-design sessions (N=10) that generated 69 concepts, the authors built functional prototypes and tested them in a within-subjects lab study (N=32) to assess perception, attention, and understanding using questionnaires and eye-tracking. Key findings show textual labels are least effective, while Chatbot provides deepest information; Role-based timelines aid overview in primarily human articles, and Task-based timelines shift perceptions toward human involvement in primarily AI articles. The work offers design guidelines, reveals how disclosures can influence readers’ perceptions, and lays groundwork for nuanced AI provenance signals across journalism and other domains, with implications for policy, literacy, and trust in blended human–AI workflows.

Abstract

Within journalistic editorial processes, disclosing AI usage is currently limited to simplistic labels, which misses the nuance of how humans and AI collaborated on a news article. Through co-design sessions (N=10), we elicited 69 disclosure designs and implemented four prototypes that visually disclose human-AI collaboration in journalism. We then ran a within-subjects lab study (N=32) to examine how disclosure visualizations (Textual, Role-based Timeline, Task-based Timeline, Chatbot) and collaboration ratios (Primarily Human vs. Primarily AI) influenced visualization perceptions, gaze patterns, and post-experience responses. We found that textual disclosures were least effective in communicating human-AI collaboration, whereas Chatbot offered the most in-depth information. Furthermore, while role-based timelines amplified AI contribution in primarily human articles, task-based timeline shifted perceptions toward human involvement in primarily AI articles. We contribute Human-AI collaboration disclosure visualizations and their evaluation, and cautionary considerations on how visualizations can alter perceptions of AI's actual role during news article creation.

Figures (30)

• Figure 1: The two part study approach with contributions outlined in (bold) blue.
• Figure 2: Screenshot of a filled out sensitizing booklet in Miro, as part of our preparation for the co-design session.
• Figure 3: In-person co-design session setup.
• Figure 4: Example ideation template for co-design session
• Figure 5: Two examples of the results of the co-design sessions, drawn on the ideation template. The example on the left was created online in Miro, the example on the right was digitized from the in-person session.