HistoryPalette: Supporting Exploration and Reuse of Past Alternatives in Image Generation and Editing

TL;DR

HistoryPalette tackles the challenge of managing and reusing past design alternatives in prompt-driven image generation. It introduces a semantic history framework with three palettes (Position, Concept, Time) plus a version timeline and a generation view, underpinned by an inpainting pipeline, automated filtering, and clustering to organize alternatives. Two exploratory studies with creative professionals and client collaborators show that non-linear palettes support faster discovery, reuse, and collaborative understanding of design history, while a technical evaluation confirms reliable filtering of low-quality generations. The work demonstrates that semantically organized histories can de-risk generation, improve workflow flow, and facilitate design communication, with potential to generalize to other domains and cross-project reuse in the future.

Abstract

All creative tasks require creators to iteratively produce, select, and discard potentially useful ideas. Now, creativity tools include generative AI features (e.g., Photoshop Generative Fill) that increase the number of alternatives creators consider due to rapid experiments with text prompts and random generations. Creators use tedious manual systems for organizing their prior ideas by saving file versions or hiding layers, but they lack the support they want for reusing prior alternatives in personal work or in communication with others. We present HistoryPalette, a system that supports exploration and reuse of prior designs in generative image creation and editing. Using HistoryPalette, creators and their collaborators explore a "palette" of prior design alternatives organized by spatial position, topic category, and creation time. HistoryPalette enables creators to quickly preview and reuse their prior work. In creative professional and client collaborator user studies, participants generated and edited images by exploring and reusing past design alternatives with HistoryPalette.

Figures (10)

• Figure 1: HistoryPalette enables rapid exploration and reuse of past design alternatives (circles). Users use the generation view to edit their image by selectively generating new content onto the canvas. HistoryPalette captures design alternatives created through edits and stores them for later exploration. Using the palettes (Position Palette, Concept Palette, and Time Palette) users can explore past design alternatives by rapidly previewing them in the place they originally occurred. HistoryPalette groups alternatives in clusters and classifies previous unsuccessful generations to be bad past alternatives (grayed-out alternatives)
• Figure 2: Overview of palettes in our system: the position palette (left), the concept palette (middle), and the time palette (right). The version timeline of full project versions appears at the bottom.
• Figure 3: Users interact with the canvas by selecting a region and entering a prompt, such as "modern house." HistoryPalette presents 4 generated design alternatives per request using Stable Diffusion XL and ControlNet. Each design alternative is post-processed for HistoryPalette's palettes. The positional encoding organizes design alternatives by location for the position palette, while clustering by concept groups design alternatives into relevant categories like "Houses" in the concept palette.
• Figure 4: Creative professionals and client collaborators interactions with HistoryPalette and the palettes. While creative professionals used the palettes more frequently as the project progressed, client collaborators first got acquainted with the projects by previewing via the version timeline.
• Figure 5: E1 reused a past design alternative of a golf course after making changes to the surrounding elements (here: generating a modern villa on the opposite side of the image).