AIdeation: Designing a Human-AI Collaborative Ideation System for Concept Designers

TL;DR

AIdeation addresses the real-world needs of concept designers by delivering a human-centered, iterative ideation system that supports breadth, depth, and flexible refinement. Through formative and summative studies with professionals and a field deployment in production studios, the approach demonstrates significant gains in creativity, idea diversity, and workflow efficiency, while highlighting barriers such as transparency, controllability, and model diversity. The work contributes a modular, multi-model pipeline that integrates research, brainstorming, and refinement, offering practical guidance for deploying GenAI in iterative design contexts. The findings suggest that when GenAI tools are domain-tuned, transparent, and controllable, they can meaningfully augment early-stage design processes in entertainment industries.

Abstract

Concept designers in the entertainment industry create highly detailed, often imaginary environments for movies, games, and TV shows. Their early ideation phase requires intensive research, brainstorming, visual exploration, and combination of various design elements to form cohesive designs. However, existing AI tools focus on image generation from user specifications, lacking support for the unique needs and complexity of concept designers' workflows. Through a formative study with 12 professional designers, we captured their workflows and identified key requirements for AI-assisted ideation tools. Leveraging these insights, we developed AIdeation to support early ideation by brainstorming design concepts with flexible searching and recombination of reference images. A user study with 16 professional designers showed that AIdeation significantly enhanced creativity, ideation efficiency, and satisfaction (all p<.01) compared to current tools and workflows. A field study with 4 studios for 1 week provided insights into AIdeation's benefits and limitations in real-world projects. After the completion of the field study, two studios, covering films, television, and games, have continued to use AIdeation in their commercial projects to date, further validating AIdeation's improvement in ideation quality and efficiency.

Figures (10)

• Figure 1: The figure showcases designs from concept to final product, including four well-known projects: (a) a scene from Star Wars, (b) characters from DC Comics (Harley Quinn, the Joker, and the Penguin), (c) a prop from Mad Max: Fury Road, and (d) a creature from Genshin Impact. This demonstrates the critical role of concept designers in shaping the creative vision from the earliest production stages to the final product
• Figure 2: A typical workflow for an environment concept designer begins with receiving the design specification from the art director or client. The designer then (a) determines a potential design direction and enters the iterative ideation cycle, which includes (b) researching based on the task, and (c) brainstorming innovative ideas. Once some suitable design ideas are formed, (d) both sketches and references are presented to art directors or clients for feedback. Upon approval, (e) they refine the sketch into a polished, detailed design, which is then shared with other teams, such as the CG team.
• Figure 3: Based on our formative study, concept designers categorize references into three types: (a) Hero (or Main) References: These align closely with the designer's creative vision, conveying the overall story, mood, or design, and are crucial for guiding the project. (b) Detailed Supporting References: These provide specific details, like structure or texture, helping designers accurately implement finer aspects of the design. (c) Miscellaneous References: These cover a range of purposes, including lighting, atmosphere, and color palette, supporting various design elements.
• Figure 4: The main interface of AIdeation includes (a) the Ideas Overview Panel, displaying all brainstormed design ideas as images with titles based on user input. Users can select an idea to view in (b) the Idea Detail Panel, which provides detailed information on the selected idea. (b1) The left sidebar lists key elements extracted from the idea, categorized into six groups as keywords. Users can select a keyword to view related search results in (b2). (b3) The right panel allows users to refine the idea by combining it with references or by instruction. (b4) Below the current idea, its origin is shown; in this case, the idea was generated by combining "Idea 4" and a colorful sofa.
• Figure 5: Technical pipeline of AIdeation: (a) The user’s input image is captioned by a vision model and processed by Idea Generation GPT, which integrates instructions and creative score to generate design ideas description. This idea is then converted into keywords, and DALL-E 3 generates an image with the idea description. (b) User-selected keywords initiate a Bing Image Search, returning a set of relevant images. When the user wants to refine the idea, (c) a selected reference is captioned by a vision model and processed by Combine Reference GPT, merging it with the original idea to create modified designs based on the creative score. (d) In contrast, AIdeation also supports refining ideas by instruction. The original idea and user instructions are processed by Refine by Instruction GPT, along with the creative score, to generate additional refined ideas.