HAIGEN: Towards Human-AI Collaboration for Facilitating Creativity and Style Generation in Fashion Design

TL;DR

This work introduces HAIGEN (Human-AI Collaboration for GENeration), an efficient fashion design system for Human-AI collaboration developed to aid designers that effectively safeguards design privacy by avoiding the need to upload personalized data from local designers.

Abstract

The process of fashion design usually involves sketching, refining, and coloring, with designers drawing inspiration from various images to fuel their creative endeavors. However, conventional image search methods often yield irrelevant results, impeding the design process. Moreover, creating and coloring sketches can be time-consuming and demanding, acting as a bottleneck in the design workflow. In this work, we introduce HAIGEN (Human-AI Collaboration for GENeration), an efficient fashion design system for Human-AI collaboration developed to aid designers. Specifically, HAIGEN consists of four modules. T2IM, located in the cloud, generates reference inspiration images directly from text prompts. With three other modules situated locally, the I2SM batch generates the image material library into a certain designer-style sketch material library. The SRM recommends similar sketches in the generated library to designers for further refinement, and the STM colors the refined sketch according to the styles of inspiration images. Through our system, any designer can perform local personalized fine-tuning and leverage the powerful generation capabilities of large models in the cloud, streamlining the entire design development process. Given that our approach integrates both cloud and local model deployment schemes, it effectively safeguards design privacy by avoiding the need to upload personalized data from local designers. We validated the effectiveness of each module through extensive qualitative and quantitative experiments. User surveys also confirmed that HAIGEN offers significant advantages in design efficiency, positioning it as a new generation of aid-tool for designers.

Figures (14)

• Figure 1: (a) Three distinct methods of deriving sketches from a provided clothing image are presented, highlighting noticeable stylistic differences among them. (b) Highlights three issues that emerge during the sketch coloring process, influencing the designer's overall creative workflow. (c) Our approach distinguishes itself from previous methods used for searching inspirations. With our method, it can generate images that closely align with the designer's inner thoughts based on detailed text descriptions.
• Figure 2: Basic information about the respondents.
• Figure 3: Gain insights into the current landscape of fashion designers and unearth their specific needs for AI tools.
• Figure 4: An Overview of our HAIGEN system. The left side (orange) illustrates the designer's design process. On the right side, the Text-to-Image Cloud Module (blue) is deployed in the cloud, while the Image-to-Sketch Local Module, Sketch Recommendation Module, and Style Transfer Module (green) are deployed locally.
• Figure 5: The illustration of the Stable Diffusion Model-based Text-to-Image Cloud Module. We initially utilize the VAE kingma2013auto model to map the input sample $x$ into the latent space for diffusion training. To expedite training and exert control over the style and details of the denoising process, we freeze the initial parameters of the SD model and incorporate LoRA hu2021lora and ControlNet zhang2023adding.