Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools

TL;DR

AI writing tools increasingly fragment the writing surface, prompting a design theory gap that this paper addresses by transferring the concept of literary Collage into UI and interaction design. It introduces Collage as an analytical, constructive, and critical lens with four facets—fragmentation, juxtaposed voices, multi-source material, and editorial role changes—and applies it to 36 tools to extract design patterns and propose new UI explorations. The work contributes a formal definition of Collage in AI writing tools, an operational framework with design patterns, and constructive explorations that promote transparency, revisability, and accountable use of AI-generated text. It also engages a critical perspective on language provenance, prompts, and the influence of AI on authorship, urging designers to consider historical critiques alongside current capabilities to shape responsible, reflective writing tools.

Abstract

This essay proposes and explores the concept of Collage for the design of AI writing tools, transferred from avant-garde literature with four facets: 1) fragmenting text in writing interfaces, 2) juxtaposing voices (content vs command), 3) integrating material from multiple sources (e.g. text suggestions), and 4) shifting from manual writing to editorial and compositional decision-making, such as selecting and arranging snippets. The essay then employs Collage as an analytical lens to analyse the user interface design of recent AI writing tools, and as a constructive lens to inspire new design directions. Finally, a critical perspective relates the concerns that writers historically expressed through literary collage to AI writing tools. In a broad view, this essay explores how literary concepts can help advance design theory around AI writing tools. It encourages creators of future writing tools to engage not only with new technological possibilities, but also with past writing innovations.

Figures (6)

• Figure 1: Using Collage as a lens on UI design highlights the use of different kinds of text fragments. Concretely, this figure shows the identified patterns of text display and interaction in UIs for writing with AI. We distinguish four text types (legend, bottom left), which result from combining two text sources (human vs AI, see \ref{['sec:collage_sources']}) and two voices (diegetic i.e. "content text" vs non-diegetic i.e. "text about text", see \ref{['sec:collage_voices']}). With these elements, the figure reveals four aspects of displaying text (A, B, C, D), as well as two aspects of text interaction (E, F). See \ref{['sec:ui_patterns']} for a detailed description. Best viewed in colour.
• Figure 2: Illustration of using the patterns (\ref{['fig:ui_patterns']}) to construct new tool designs. Starting from a traditional single writing area (top left), we morph the text fragments displayed in the UI into other (sub)divisions, arrangements, voices, and sources. The arrows and action words in between the designs indicate what is changed, while the "titles" below each design shortly describe a tool idea consistent with this way of displaying text. Black arrows on the designs indicate the (predominant) intended relationship between the text fragments (cf. \ref{['fig:ui_patterns']}D); other relationships might also be possible. See \ref{['sec:ui_morphing']} for details and \ref{['sec:appendix_example_morphs']} for descriptions of all shown design ideas.
• Figure 3: Inspired by text fragmentation in Collage, this UI design idea (A) keeps and shows the seams between text fragments written by the user (black) and those suggested by the AI (blue). In this way, the UI turns the user's past suggestion choices into interactive objects: Clicking on one such object (B) reveals the alternative suggestions that the user rejected back then. This design facilitates transparency around human vs AI writing (e.g. to allow writers to double-check these AI-provided text pieces for accidental plagiarism), may help communicate one writer's intention to co-authors in collaborative writing, and could address "agony of choice" situations if the user likes multiple suggested options, as no option is lost. Users can also retroactively change their mind by selecting a different alternative. In a more advanced tool design, this could prompt an AI-rewrite of the subsequent story parts, to match the semantics of the updated suggestion choice. See \ref{['sec:design_exploration1_suggestions']} for a detailed description.
• Figure 4: Inspired by the juxtaposition of text from different voices in Collage, this design idea (A) shows the seams between text fragments written as part of the draft (black) and those entered to prompt the AI (blue). In this way, the UI turns the user's past prompts into interactive objects: Clicking on one such object (B) reveals the corresponding text generated by the AI (which the user may also edit just as any other text). This design allows users to further benefit from the effort they put into their prompts, since prompt texts do not disappear. Instead, the prompt texts serve as summaries of the respective parts of the draft. This might be useful to gain an overview when coming back to a writing project or when going through the latest version shared by a collaborator. It might also facilitate efficient context understanding for editing tasks, without having to read everything. See \ref{['sec:design_exploration1_prompts']} for a detailed description.
• Figure 5: Inspired by rejectingCollage, this design idea (A) aims to minimise the number of text fragments displayed at a time. The result is a "focus mode"', in which previously entered lines disappear while typing (e.g. akin to submitting a line in a chat app UI, which scrolls earlier messages out of view). In line with the "freewriting" method li2007freewriting, this might help users with pushing on with the draft, instead of perfecting their wording or otherwise getting tempted by (premature) revisions. AI processes the user's sentences in the background to fix typos and grammar, remove redundancies, extend fragments into complete sentences, and so on. In this way, the user receives a cleaned-up version when leaving this writing mode (B). Lines with AI changes could be marked (e.g. by blue dots) to provide transparency about AI revisions, without visually fragmenting the text display with word-based highlights, as was the case in \ref{['fig:design_exploration1']}. See \ref{['sec:design_exploration2']} for a detailed description.