Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization

TL;DR

Patchview introduces Generative Dust and Magnet (GD&M) visualization to help writers sensemake, steer, and correct LLM-generated world elements during worldbuilding. It integrates a visual metaphor where elements (dust) are attracted to user-defined concept magnets, enabling continuous, nuanced steering in magnet space and feedback corrections to align AI outputs with user intent. A web-based tool combines a list and a view module to support sensemaking across multiple concept spaces, with prompting strategies using Claude models. A user study with nine participants demonstrates improved sensemaking and intuitive visual steering, while corrections to AI behavior yielded limited realignment, highlighting both promise and the need for further technical refinement. The work discusses implications for visual interfaces in prompting, model alignment, and future enhancements, such as timelines and consistency management.

Abstract

Large language models (LLMs) can help writers build story worlds by generating world elements, such as factions, characters, and locations. However, making sense of many generated elements can be overwhelming. Moreover, if the user wants to precisely control aspects of generated elements that are difficult to specify verbally, prompting alone may be insufficient. We introduce Patchview, a customizable LLM-powered system that visually aids worldbuilding by allowing users to interact with story concepts and elements through the physical metaphor of magnets and dust. Elements in Patchview are visually dragged closer to concepts with high relevance, facilitating sensemaking. The user can also steer the generation with verbally elusive concepts by indicating the desired position of the element between concepts. When the user disagrees with the LLM's visualization and generation, they can correct those by repositioning the element. These corrections can be used to align the LLM's future behaviors to the user's perception. With a user study, we show that Patchview supports the sensemaking of world elements and steering of element generation, facilitating exploration during the worldbuilding process. Patchview provides insights on how customizable visual representation can help sensemake, steer, and align generative AI model behaviors with the user's intentions.

Figures (20)

• Figure 1: Compared to dust and magnet visualization, generative dust and magnet replaces data elements (dust particles) and variables (magnets) with generated data elements and concepts, respectively. In generative dust and magnet, the distance between a magnet and a dust particle indicates the intensity of relevance between them.
• Figure 2: Input-Output schemes for GD&M Interactions
• Figure 3: Configuration interactions for evaluation support in generative dust and magnet. As the user adds, removes, edits, and moves concepts according to how they want to organize elements and concepts, the positions of data elements get updated.
• Figure 4: Patchview interface. a) View module visualizes world elements in relation to concepts of the user’s interest. Specific interactions are shown in Figure \ref{['fig:teaser']}. b) List module lists world elements as notes (b-4). This module allows users to generate elements by clicking buttons for different element types (b-1) or by prompting an LLM with specific natural-language instructions (b-5). The user can steer generation with a view interface (as in Figure \ref{['fig:teaser']}b) by entering the steering mode with a toggle switch (b-2). They can also manually create notes (b-3). c) The user can see a list of existing views by clicking the Views button and create a new view with the + button.
• Figure 5: Possible inputs to generate elements.