Creative Blends of Visual Concepts

TL;DR

This work introduces Creative Blends, a metaphor-informed AI system that maps abstract concepts to concrete objects via commonsense reasoning and combines them through attribute-based similarity to generate diverse visual blends. It integrates LLMs, ConceptNet, and CLIP-based similarity with sentiment analysis to produce structured blending prompts, which are then realized as images with DALL·E 3, all presented in an interpretable visualization interface. A formative study informs design goals, while a within-subject evaluation with 24 participants demonstrates improved usability, reduced cognitive load, and enhanced creativity and metaphorical richness compared to a GPT-3.5+DALL·E 3 baseline. The findings underscore the potential of semantic-attribute grounded blending for ideation and offer insights into user preferences, generalization, and the integration of generative AI in design workflows. Limitations include scope to ideation and short-term evaluation, with future work focusing on longer studies, prompt-editing controls, and refinement-stage support to further empower designers."

Abstract

Visual blends combine elements from two distinct visual concepts into a single, integrated image, with the goal of conveying ideas through imaginative and often thought-provoking visuals. Communicating abstract concepts through visual blends poses a series of conceptual and technical challenges. To address these challenges, we introduce Creative Blends, an AI-assisted design system that leverages metaphors to visually symbolize abstract concepts by blending disparate objects. Our method harnesses commonsense knowledge bases and large language models to align designers' conceptual intent with expressive concrete objects. Additionally, we employ generative text-to-image techniques to blend visual elements through their overlapping attributes. A user study (N=24) demonstrated that our approach reduces participants' cognitive load, fosters creativity, and enhances the metaphorical richness of visual blend ideation. We explore the potential of our method to expand visual blends to include multiple object blending and discuss the insights gained from designing with generative AI.

Figures (20)

• Figure 1: The Creative Blends system leverages textual user input to generate visual blends, offering a platform for creative exploration. The system broadens the spectrum of design possibilities by mapping abstract concepts -- highlighted in blue and orange colors -- onto tangible physical objects through constructed metaphorical associations. These mapped objects are then seamlessly integrated based on shared attributes, resulting in conceptually meaningful and visually cohesive blends.
• Figure 2: Creative Blends operates through a multi-stage pipeline. The initial stage involves concept inference to identify relevant objects and their attributes. Subsequently, a similarity-based selection process empowers users to choose suitable object and attribute combinations. The system then explores potential blending schemes and synthesizes corresponding prompts for the T2I model, culminating in iterative image generation based on the selected prompts to support the ideation process.
• Figure 3: The user interface of Creative Blends showcases an example of generated results for "global warming". The interface consists of four distinct modules: the expression input module (a), the prompt exploration module (b, c, d), the visual blend exploration module (e), and the similarity visualization module (f).
• Figure 4: The scheme generation prompt is structured into five key modules: (a) system setup, (b) task definition, (c) user input, (d) task execution process, and (e) results format demonstration. The (d) task execution process module outlines the methods and rationales for considering potential blending schemes. The results, processed by the (e) results format demonstration module, are returned in a standardized JSON format, ensuring compatibility with downstream processes.
• Figure 5: The final prompt is composed of four distinct modules: (1) objects, (2) attributes, (3) schemes, and (4) considerations. The schemes and metaphorical themes (marked in capital letters with a grey background), essential elements of the prompt, are dynamically generated by the GPT in response to the scheme and metaphor generation prompts.