StylizedGS: Controllable Stylization for 3D Gaussian Splatting

TL;DR

StylizedGS presents a fast, controllable 3D Gaussian Splatting-based stylization method that transfers 2D style images into 3D scenes by jointly refining geometry and color. It introduces a filter-based refinement to remove floaters, a nearest-neighbor feature matching loss to capture fine style details, and a depth-preserving regularization to maintain geometry. The approach enables user-controlled perceptual factors—color, style scale, and spatial regions—via tailored losses and masks, yielding coherent, view-consistent stylizations with sub-minute training and real-time rendering. Across multiple datasets and styles, StylizedGS demonstrates superior stylization quality and efficiency compared to NeRF- and 3DGS-based baselines, with robust ablations supporting the design choices. The work advances interactive, image-based 3D stylization by leveraging explicit 3DGS representations and perceptual controls for creative exploration.

Abstract

As XR technology continues to advance rapidly, 3D generation and editing are increasingly crucial. Among these, stylization plays a key role in enhancing the appearance of 3D models. By utilizing stylization, users can achieve consistent artistic effects in 3D editing using a single reference style image, making it a user-friendly editing method. However, recent NeRF-based 3D stylization methods encounter efficiency issues that impact the user experience, and their implicit nature limits their ability to accurately transfer geometric pattern styles. Additionally, the ability for artists to apply flexible control over stylized scenes is considered highly desirable to foster an environment conducive to creative exploration. To address the above issues, we introduce StylizedGS, an efficient 3D neural style transfer framework with adaptable control over perceptual factors based on 3D Gaussian Splatting representation. We propose a filter-based refinement to eliminate floaters that affect the stylization effects in the scene reconstruction process. The nearest neighbor-based style loss is introduced to achieve stylization by fine-tuning the geometry and color parameters of 3DGS, while a depth preservation loss with other regularizations is proposed to prevent the tampering of geometry content. Moreover, facilitated by specially designed losses, StylizedGS enables users to control color, stylized scale, and regions during the stylization to possess customization capabilities. Our method achieves high-quality stylization results characterized by faithful brushstrokes and geometric consistency with flexible controls. Extensive experiments across various scenes and styles demonstrate the effectiveness and efficiency of our method concerning both stylization quality and inference speed.

Figures (26)

• Figure 1: Stylization Results. Given a 2D style image, the proposed StylizedGS method can stylize the pre-trained 3D Gaussian Splatting to match the desired style with detailed geometric features and satisfactory visual quality within a few minutes. We also enable users to control several perceptual factors, such as color, the style pattern size (scale), and the stylized regions (spatial), during the stylization to enhance the customization capabilities.
• Figure 1: Additional color control stylization results. Our approach facilitates versatile color management in stylized outputs, allowing users to retain the scene's original hues or apply distinct color schemes from alternative style images. Users can choose to transfer the entire style, only the pattern style, or a mix of arbitrary patterns and color styles.
• Figure 2: StylizedGS Pipeline. We first reconstruct a photo-realistic 3DGS $G_{\theta}^{rec}$ from multi-view input. Following this, color matching with the style image is performed, accompanied by the filter-based refinement to preemptively address potential artifacts. During optimization, we employ multiple loss terms to capture detailed local style structures and preserve geometric attributes. Users can flexibly control color, scale, and spatial attributes during stylization through customizable loss terms. Once this stylization is done, we can obtain consistent free-viewpoint stylized renderings.
• Figure 2: Additional scale control stylization results. Our method enables users to flexibly control the scale of basic style elements, such as adjusting the density of colorful blocks, as demonstrated in the first row.
• Figure 3: Color Control Results. Our approach facilitates versatile color management in stylized outputs, allowing users to retain the scene's original hues or apply distinct color schemes from alternative style images. Users can choose to transfer the entire style, only the pattern style, or a mix of arbitrary patterns and color styles.