DiffStyle3D: Consistent 3D Gaussian Stylization via Attention Optimization

TL;DR

DiffStyle3D tackles the challenge of multi-view-consistent 3D Gaussian Splatting style transfer by moving optimization into the diffusion model's latent space. It introduces an Attention-Aware Loss that transfers style while preserving content through self-attention feature alignment, and a Geometry-Guided Multi-View Consistency mechanism that injects geometric cues into attention to model cross-view correspondences. A geometry-aware mask further reduces redundant updates in overlapping regions, improving consistency across views. Extensive experiments show DiffStyle3D outperforms state-of-the-art methods in stylization quality, content preservation, and multi-view coherence with efficient training on standard GPUs.

Abstract

3D style transfer enables the creation of visually expressive 3D content, enriching the visual appearance of 3D scenes and objects. However, existing VGG- and CLIP-based methods struggle to model multi-view consistency within the model itself, while diffusion-based approaches can capture such consistency but rely on denoising directions, leading to unstable training. To address these limitations, we propose DiffStyle3D, a novel diffusion-based paradigm for 3DGS style transfer that directly optimizes in the latent space. Specifically, we introduce an Attention-Aware Loss that performs style transfer by aligning style features in the self-attention space, while preserving original content through content feature alignment. Inspired by the geometric invariance of 3D stylization, we propose a Geometry-Guided Multi-View Consistency method that integrates geometric information into self-attention to enable cross-view correspondence modeling. Based on geometric information, we additionally construct a geometry-aware mask to prevent redundant optimization in overlapping regions across views, which further improves multi-view consistency. Extensive experiments show that DiffStyle3D outperforms state-of-the-art methods, achieving higher stylization quality and visual realism.

Figures (9)

• Figure 1: Our method enables high-quality 3D stylization across diverse styles for both scenes and objects.
• Figure 2: Overview of DiffStyle3D. We introduce an Attention-Aware Loss that enables style transfer while preserving content. To model multi-view correspondences, we derive a explicit geometry guidance from camera parameters and depth maps and incorporate it into Self-Attention (SA) to form Geometry-Guided Attention (GGA). Additionally, a geometry-aware mask $\mathcal{M}_G$ restricts optimization to non-overlapping regions, further improving multi-view consistency.
• Figure 3: Results with different stylization signals. We conduct experiments using a fixed viewpoint of the 3D scene. Directly using the attention outputs of the style image as stylization signals leads to severe content leakage.
• Figure 4: Results obtained using different timestep during optimization. Random denotes randomly sampled timestep throughout the optimization process, while decreasing simulates the diffusion process by progressively decreasing the time step from $T$ to 0.
• Figure 5: Qualitative comparison of different methods on scene-level datasets. Our approach achieves superior style transfer while better preserving the original content. The red boxes highlight clear differences, the details of which are further compared in Fig. \ref{['fig:comparison_detail']}.