Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Style Transfer Dataset: What Makes A Good Stylization?

Victor Kitov, Valentin Abramov, Mikhail Akhtyrchenko

TL;DR

The paper tackles the challenge of reproducible evaluation for image style transfer by introducing a large, permissively licensed dataset with 50 content images, 50 style images, and four style-size scales, yielding 10,000 stylizations rated by three annotators (totaling 30,000 ratings). Stylizations are generated with ArtFlow, recolored in LAB to separate color transfer from texture, and evaluated to identify quantitative and qualitative factors driving perceived quality. Key findings reveal that color and brightness diversity, edge preservation, and, most strongly, style size critically influence ratings, with notable sensitivity to face reproduction and content characteristics. The work provides practical recommendations and an evaluation tool to enable automated, comparable assessments across studies, advancing the development and tuning of style transfer methods for real-world use.

Abstract

We present a new dataset with the goal of advancing image style transfer - the task of rendering one image in the style of another image. The dataset covers various content and style images of different size and contains 10.000 stylizations manually rated by three annotators in 1-10 scale. Based on obtained ratings, we find which factors are mostly responsible for favourable and poor user evaluations and show quantitative measures having statistically significant impact on user grades. A methodology for creating style transfer datasets is discussed. Presented dataset can be used in automating multiple tasks, related to style transfer configuration and evaluation.

Style Transfer Dataset: What Makes A Good Stylization?

TL;DR

The paper tackles the challenge of reproducible evaluation for image style transfer by introducing a large, permissively licensed dataset with 50 content images, 50 style images, and four style-size scales, yielding 10,000 stylizations rated by three annotators (totaling 30,000 ratings). Stylizations are generated with ArtFlow, recolored in LAB to separate color transfer from texture, and evaluated to identify quantitative and qualitative factors driving perceived quality. Key findings reveal that color and brightness diversity, edge preservation, and, most strongly, style size critically influence ratings, with notable sensitivity to face reproduction and content characteristics. The work provides practical recommendations and an evaluation tool to enable automated, comparable assessments across studies, advancing the development and tuning of style transfer methods for real-world use.

Abstract

We present a new dataset with the goal of advancing image style transfer - the task of rendering one image in the style of another image. The dataset covers various content and style images of different size and contains 10.000 stylizations manually rated by three annotators in 1-10 scale. Based on obtained ratings, we find which factors are mostly responsible for favourable and poor user evaluations and show quantitative measures having statistically significant impact on user grades. A methodology for creating style transfer datasets is discussed. Presented dataset can be used in automating multiple tasks, related to style transfer configuration and evaluation.

Paper Structure

This paper contains 15 sections, 1 equation, 10 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset Creation Process
  4. Content and Style Images
  5. Stylization Creation
  6. Style Recoloring
  7. Labeling Process
  8. General Analysis of Ratings
  9. Impact Factors
  10. Color Diversity and Sharpness
  11. Texture Blur and Edges Reproduction
  12. Face Reproduction
  13. Style Size and Characteristics
  14. Recommendations
  15. Conclusion

Figures (10)

  • Figure 1: Style transfer task: redraw content image in the style of the style image transferring style colors as well (first row) and preserving original content colors (second row).
  • Figure 2: Style transfer using style image with different size: $700^2$ pixels (upper row) and $300^2$ pixels (lower row).
  • Figure 3: Common border artifact present in stylizations by ArtFlow style transfer method.
  • Figure 4: Rating distribution for 3 annotators and mean rating.
  • Figure 5: Distribution of standard deviations of ratings.
  • ...and 5 more figures