Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas

Jia Wei Sii, Chee Seng Chan

TL;DR

Gorgeous tackles the limitations of makeup transfer by enabling creative character makeup generation from arbitrary inspirations without requiring a face in the reference images. Built on a diffusion framework, it combines three modules—MaFor (makeup formatting via ControlNet), CSL (learned style embeddings from 3–5 references), and MaIP (face-focused inpainting)—to apply makeup that respects facial identity while translating non-facial and thematic cues into wearable makeup. Extensive qualitative and quantitative evaluations, including user studies, show that Gorgeous yields distinctive, thematically aligned character makeup with favorable FID, CSD, and DreamSIM scores compared to state-of-the-art baselines. The approach broadens the creative toolkit for visual storytelling, enabling rapid, flexible, and identity-preserving makeup design guided by diverse inspirations.

Abstract

Contemporary makeup transfer methods primarily focus on replicating makeup from one face to another, considerably limiting their use in creating diverse and creative character makeup essential for visual storytelling. Such methods typically fail to address the need for uniqueness and contextual relevance, specifically aligning with character and story settings as they depend heavily on existing facial makeup in reference images. This approach also presents a significant challenge when attempting to source a perfectly matched facial makeup style, further complicating the creation of makeup designs inspired by various story elements, such as theme, background, and props that do not necessarily feature faces. To address these limitations, we introduce $Gorgeous$, a novel diffusion-based makeup application method that goes beyond simple transfer by innovatively crafting unique and thematic facial makeup. Unlike traditional methods, $Gorgeous$ does not require the presence of a face in the reference images. Instead, it draws artistic inspiration from a minimal set of three to five images, which can be of any type, and transforms these elements into practical makeup applications directly on the face. Our comprehensive experiments demonstrate that $Gorgeous$ can effectively generate distinctive character facial makeup inspired by the chosen thematic reference images. This approach opens up new possibilities for integrating broader story elements into character makeup, thereby enhancing the narrative depth and visual impact in storytelling.

Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas

TL;DR

Gorgeous tackles the limitations of makeup transfer by enabling creative character makeup generation from arbitrary inspirations without requiring a face in the reference images. Built on a diffusion framework, it combines three modules—MaFor (makeup formatting via ControlNet), CSL (learned style embeddings from 3–5 references), and MaIP (face-focused inpainting)—to apply makeup that respects facial identity while translating non-facial and thematic cues into wearable makeup. Extensive qualitative and quantitative evaluations, including user studies, show that Gorgeous yields distinctive, thematically aligned character makeup with favorable FID, CSD, and DreamSIM scores compared to state-of-the-art baselines. The approach broadens the creative toolkit for visual storytelling, enabling rapid, flexible, and identity-preserving makeup design guided by diverse inspirations.

Abstract

Contemporary makeup transfer methods primarily focus on replicating makeup from one face to another, considerably limiting their use in creating diverse and creative character makeup essential for visual storytelling. Such methods typically fail to address the need for uniqueness and contextual relevance, specifically aligning with character and story settings as they depend heavily on existing facial makeup in reference images. This approach also presents a significant challenge when attempting to source a perfectly matched facial makeup style, further complicating the creation of makeup designs inspired by various story elements, such as theme, background, and props that do not necessarily feature faces. To address these limitations, we introduce , a novel diffusion-based makeup application method that goes beyond simple transfer by innovatively crafting unique and thematic facial makeup. Unlike traditional methods, does not require the presence of a face in the reference images. Instead, it draws artistic inspiration from a minimal set of three to five images, which can be of any type, and transforms these elements into practical makeup applications directly on the face. Our comprehensive experiments demonstrate that can effectively generate distinctive character facial makeup inspired by the chosen thematic reference images. This approach opens up new possibilities for integrating broader story elements into character makeup, thereby enhancing the narrative depth and visual impact in storytelling.
Paper Structure (42 sections, 7 equations, 30 figures, 4 tables)

This paper contains 42 sections, 7 equations, 30 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Facial Makeup
  4. Style Transfer
  5. Other Diffusion Models
  6. Image Inpainting
  7. Methodology
  8. Makeup Formatting (MaFor) Module
  9. Paired Data Preparation.
  10. ControlNet Training.
  11. Character Settings Learning (CSL) Module
  12. Makeup Inpainting Pipeline (MaIP)
  13. Note.
  14. Experiments
  15. Datasets and Evaluation Metrics
  16. ...and 27 more sections

Figures (30)

  • Figure 1: Provide us with any reference images of your desired character settings (e.g., war, sunflower), and our $Gorgeous$ will transform them into a creative and unique character makeup design that enriches your visual storytelling!
  • Figure 2: Overall $Gorgeous$ architecture. Given a set of inspirational reference images, these images are processed to extract and embed inspirational elements into a placeholder token. This token is then utilized within the MaIP as a simple textual guide to generate character-specific makeup designs. Throughout this process, the generation is consistently overseen by our pretrained MaFor module, ensuring that the outputs strictly adhere to makeup designs without deviating into unrelated elements.
  • Figure 3: (A) showcases a collection of inspiration reference images for eight distinct character settings. These are divided into two categories: Style 1 (a) to (d), which include reference images featuring human faces, while Style 2 (a) to (d), which comprise images without human faces. Panels (B) and (C) display our qualitative results for Style 1 and Style 2, respectively, in comparison with state-of-the-art makeup transfer methods (i.e., EleGANt, SSAT, BeautyREC) and a style transfer method (i.e., InST). Outputs that directly replicate the style from the reference images are marked with a blue circle, while those that are creatively inspired by the styles are indicated with an orange star.
  • Figure 4: Other qualitative results compared with existing state-of-the-art text-guided image-to-image generation/editing methods to evaluate their capabilities in generating your desired character facial makeups. While the image input is the Naked Face, the text prompts are listed in the figure to guide the generation.
  • Figure 5: Ablation Study
  • ...and 25 more figures