Style Brush: Guided Style Transfer for 3D Objects

TL;DR

Style Brush addresses guided 3D style transfer for textured meshes by introducing a differentiable rendering framework guided by user-provided contours. It builds a rotated style feature dictionary via edge tangent flow, applies color matching, and optimizes texture using a nearest-neighbor feature matching loss with directional constraints across multiple viewpoints. The method supports partial and multi-style applications and uses a multiscale strategy to form large-scale patterns while preserving detail, delivering coherent textures in minutes. This enables artists to concretely control stylization in 3D, achieving high-quality, direction-consistent textures that faithful to the input guidance and style images.

Abstract

We introduce Style Brush, a novel style transfer method for textured meshes designed to empower artists with fine-grained control over the stylization process. Our approach extends traditional 3D style transfer methods by introducing a novel loss function that captures style directionality, supports multiple style images or portions thereof, and enables smooth transitions between styles in the synthesized texture. The use of easily generated guiding textures streamlines user interaction, making our approach accessible to a broad audience. Extensive evaluations with various meshes, style images, and contour shapes demonstrate the flexibility of our method and showcase the visual appeal of the generated textures.

Figures (8)

• Figure 1: Overview of our method: We take a textured mesh, an additional texture with guiding lines, and a style image. Initially, we extract rotated style features by rotating the style image, passing it through a feature extractor, and assigning to each feature its directionality, thus building an angle-based feature dictionary. Next, we perform a color matching step between the style image and the original texture. Finally, we optimize the texture by rendering the mesh from multiple viewpoints, extracting features, and calculating the style loss by matching the rotated style features to the directions defined by the rendered contours. We minimize the total variation loss to suppress noise and repeat the entire process until convergence.
• Figure 2: Results generated with Style Brush for six textured meshes (columns), using as input the initial textured mesh (first row), directional guidance in the form of contours (second row), and five style exemplars (last five rows).
• Figure 3: Two examples of our method: the one on the left uses the full style image, while the one on the right applies only the features corresponding to a mask (bottom right).
• Figure 4: The results of our method for three meshes and five different styles, where two styles are combined. Here, two different style images are used on different parts of the mesh, defined by the style mask texture (indicated with orange and green). The style mask texture is given as an input to Style Brush, in addition to the original textured mesh and the guiding contours.
• Figure 5: A stylized texture generated with our method, combining three style regions, each with a different style image.