Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Models

TL;DR

The paper addresses cultural bias and data efficiency in fashion content generation by integrating LLMs with Latent Diffusion Models. It introduces a Prompt Refinement Module, a Latent Diffusion Model, and a Weak Supervision Filtering Module, with a forward–reverse diffusion process and a composite loss that includes prompt consistency and reconstruction terms. Empirical results show state-of-the-art FID/IS improvements and strong human judgments, along with ablations that confirm the value of prompt refinement and weak supervision. The approach demonstrates scalable, culturally inclusive fashion synthesis with potential impact on design prototyping, virtual try-on, and global fashion representation.

Abstract

Fashion content generation is an emerging area at the intersection of artificial intelligence and creative design, with applications ranging from virtual try-on to culturally diverse design prototyping. Existing methods often struggle with cultural bias, limited scalability, and alignment between textual prompts and generated visuals, particularly under weak supervision. In this work, we propose a novel framework that integrates Large Language Models (LLMs) with Latent Diffusion Models (LDMs) to address these challenges. Our method leverages LLMs for semantic refinement of textual prompts and introduces a weak supervision filtering module to effectively utilize noisy or weakly labeled data. By fine-tuning the LDM on an enhanced DeepFashion+ dataset enriched with global fashion styles, the proposed approach achieves state-of-the-art performance. Experimental results demonstrate that our method significantly outperforms baselines, achieving lower Frechet Inception Distance (FID) and higher Inception Scores (IS), while human evaluations confirm its ability to generate culturally diverse and semantically relevant fashion content. These results highlight the potential of LLM-guided diffusion models in driving scalable and inclusive AI-driven fashion innovation.