A Practical Style Transfer Pipeline for 3D Animation: Insights from Production R&D

TL;DR

The insights from the development process, where various options to balance quality, artist control, and workload were explored, led to several key decisions, including patch-based texture synthesis over machine learning for better control and to avoid training data issues.

Abstract

Our animation studio has developed a practical style transfer pipeline for creating stylized 3D animation, which is suitable for complex real-world production. This paper presents the insights from our development process, where we explored various options to balance quality, artist control, and workload, leading to several key decisions. For example, we chose patch-based texture synthesis over machine learning for better control and to avoid training data issues. We also addressed specifying style exemplars, managing multiple colors within a scene, controlling outlines and shadows, and reducing temporal noise. These insights were used to further refine our pipeline, ultimately enabling us to produce an experimental short film showcasing various styles.

Figures (5)

• Figure 1: The overview of the proposed style transfer pipeline.
• Figure 2: Artifacts of the tested naïve approach (i.e., perform style transfer for each color and then combine).
• Figure 3: The concept of temporal noise reduction using an additional style transfer step for advection.
• Figure 4: Additional style variations generated by our style transfer pipeline: (a) hatching and (b) pastel painting styles.
• Figure 5: Production still images from our experimental short film, highlighting various styles effectively communicating the film's atmosphere. © ARCH $\cdot$ Graphinica $\cdot$ Salamander Pictures