Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MPDS: A Movie Posters Dataset for Image Generation with Diffusion Model

Meng Xu, Tong Zhang, Fuyun Wang, Yi Lei, Xin Liu, Zhen Cui

TL;DR

For movie poster generation, a multi-condition diffusion framework is developed that takes poster prompt, poster caption, and actor image (for personalization) as inputs, yielding excellent results through the learning of a diffusion model.

Abstract

Movie posters are vital for captivating audiences, conveying themes, and driving market competition in the film industry. While traditional designs are laborious, intelligent generation technology offers efficiency gains and design enhancements. Despite exciting progress in image generation, current models often fall short in producing satisfactory poster results. The primary issue lies in the absence of specialized poster datasets for targeted model training. In this work, we propose a Movie Posters DataSet (MPDS), tailored for text-to-image generation models to revolutionize poster production. As dedicated to posters, MPDS stands out as the first image-text pair dataset to our knowledge, composing of 373k+ image-text pairs and 8k+ actor images (covering 4k+ actors). Detailed poster descriptions, such as movie titles, genres, casts, and synopses, are meticulously organized and standardized based on public movie synopsis, also named movie-synopsis prompt. To bolster poster descriptions as well as reduce differences from movie synopsis, further, we leverage a large-scale vision-language model to automatically produce vision-perceptive prompts for each poster, then perform manual rectification and integration with movie-synopsis prompt. In addition, we introduce a prompt of poster captions to exhibit text elements in posters like actor names and movie titles. For movie poster generation, we develop a multi-condition diffusion framework that takes poster prompt, poster caption, and actor image (for personalization) as inputs, yielding excellent results through the learning of a diffusion model. Experiments demonstrate the valuable role of our proposed MPDS dataset in advancing personalized movie poster generation. MPDS is available at https://anonymous.4open.science/r/MPDS-373k-BD3B.

MPDS: A Movie Posters Dataset for Image Generation with Diffusion Model

TL;DR

For movie poster generation, a multi-condition diffusion framework is developed that takes poster prompt, poster caption, and actor image (for personalization) as inputs, yielding excellent results through the learning of a diffusion model.

Abstract

Movie posters are vital for captivating audiences, conveying themes, and driving market competition in the film industry. While traditional designs are laborious, intelligent generation technology offers efficiency gains and design enhancements. Despite exciting progress in image generation, current models often fall short in producing satisfactory poster results. The primary issue lies in the absence of specialized poster datasets for targeted model training. In this work, we propose a Movie Posters DataSet (MPDS), tailored for text-to-image generation models to revolutionize poster production. As dedicated to posters, MPDS stands out as the first image-text pair dataset to our knowledge, composing of 373k+ image-text pairs and 8k+ actor images (covering 4k+ actors). Detailed poster descriptions, such as movie titles, genres, casts, and synopses, are meticulously organized and standardized based on public movie synopsis, also named movie-synopsis prompt. To bolster poster descriptions as well as reduce differences from movie synopsis, further, we leverage a large-scale vision-language model to automatically produce vision-perceptive prompts for each poster, then perform manual rectification and integration with movie-synopsis prompt. In addition, we introduce a prompt of poster captions to exhibit text elements in posters like actor names and movie titles. For movie poster generation, we develop a multi-condition diffusion framework that takes poster prompt, poster caption, and actor image (for personalization) as inputs, yielding excellent results through the learning of a diffusion model. Experiments demonstrate the valuable role of our proposed MPDS dataset in advancing personalized movie poster generation. MPDS is available at https://anonymous.4open.science/r/MPDS-373k-BD3B.

Paper Structure

This paper contains 25 sections, 4 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image-Text Datasets
  4. Vision-Language Models
  5. Text-to-Image Model
  6. MPDS Dataset
  7. Initial Data Collection
  8. Data Processing
  9. Dataset Composition
  10. Dataset Statistics
  11. Experiments
  12. The proposed multi-condition diffusion framework
  13. Implementation Datails
  14. Fine-tuning model with the MPDS
  15. Multi-condition diffusion model
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Examples of posters and actor images in MPDS.
  • Figure 3: The proposed multi-condition diffusion framework for movie poster generation.Given the input prompt $\mathbf{I}_{prompt}$, the input poster caption $\mathbf{I}_{c}$, and the input facial image of an actor $\mathbf{I}_{a}$, the proposed framework can generate images in the stype of movie posters. Concretely, the generated posters encompass: i) the scenes that align with the prompts, which consist of two parts including the movie-synopsis prompt and vision-perceptive prompt (i.e. poster scene description); ii) the role played by the designated actor; iii) the texts in poster that match the specified content of the input caption. Specifically, the poster caption $\mathbf{I}_{c}$ and actor face $\mathbf{I}_{a}$ are optional to achieve personalized poster design.
  • Figure 4: Visualizations of movie poster generation compared with existing methods, the corresponding textual prompt for the images below.
  • Figure 5: Ablation experiment results with/without different Control Branches.