Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WaSt-3D: Wasserstein-2 Distance for Scene-to-Scene Stylization on 3D Gaussians

Dmytro Kotovenko, Olga Grebenkova, Nikolaos Sarafianos, Avinash Paliwal, Pingchuan Ma, Omid Poursaeed, Sreyas Mohan, Yuchen Fan, Yilei Li, Rakesh Ranjan, Björn Ommer

TL;DR

This work leverages an explicit Gaussian Splatting representation and directly match the distributions of Gaussians between style and content scenes using the Earth Mover's Distance to achieve high-resolution 3D stylization.

Abstract

While style transfer techniques have been well-developed for 2D image stylization, the extension of these methods to 3D scenes remains relatively unexplored. Existing approaches demonstrate proficiency in transferring colors and textures but often struggle with replicating the geometry of the scenes. In our work, we leverage an explicit Gaussian Splatting (GS) representation and directly match the distributions of Gaussians between style and content scenes using the Earth Mover's Distance (EMD). By employing the entropy-regularized Wasserstein-2 distance, we ensure that the transformation maintains spatial smoothness. Additionally, we decompose the scene stylization problem into smaller chunks to enhance efficiency. This paradigm shift reframes stylization from a pure generative process driven by latent space losses to an explicit matching of distributions between two Gaussian representations. Our method achieves high-resolution 3D stylization by faithfully transferring details from 3D style scenes onto the content scene. Furthermore, WaSt-3D consistently delivers results across diverse content and style scenes without necessitating any training, as it relies solely on optimization-based techniques. See our project page for additional results and source code: $\href{https://compvis.github.io/wast3d/}{https://compvis.github.io/wast3d/}$.

WaSt-3D: Wasserstein-2 Distance for Scene-to-Scene Stylization on 3D Gaussians

TL;DR

This work leverages an explicit Gaussian Splatting representation and directly match the distributions of Gaussians between style and content scenes using the Earth Mover's Distance to achieve high-resolution 3D stylization.

Abstract

While style transfer techniques have been well-developed for 2D image stylization, the extension of these methods to 3D scenes remains relatively unexplored. Existing approaches demonstrate proficiency in transferring colors and textures but often struggle with replicating the geometry of the scenes. In our work, we leverage an explicit Gaussian Splatting (GS) representation and directly match the distributions of Gaussians between style and content scenes using the Earth Mover's Distance (EMD). By employing the entropy-regularized Wasserstein-2 distance, we ensure that the transformation maintains spatial smoothness. Additionally, we decompose the scene stylization problem into smaller chunks to enhance efficiency. This paradigm shift reframes stylization from a pure generative process driven by latent space losses to an explicit matching of distributions between two Gaussian representations. Our method achieves high-resolution 3D stylization by faithfully transferring details from 3D style scenes onto the content scene. Furthermore, WaSt-3D consistently delivers results across diverse content and style scenes without necessitating any training, as it relies solely on optimization-based techniques. See our project page for additional results and source code: .
Paper Structure (22 sections, 2 equations, 2 figures, 1 table)

This paper contains 22 sections, 2 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Initial Submission
  3. Language
  4. Template
  5. Fonts.
  6. Paper Length
  7. Paper ID
  8. Line Numbering
  9. Policies
  10. Preserving Anonymity
  11. Formatting Guidelines
  12. Headings
  13. Figures
  14. Formulas
  15. Lemmas, Propositions, and Theorems.
  16. ...and 7 more sections

Figures (2)

  • Figure 1: One kernel at $x_s$ (dotted kernel) or two kernels at $x_i$ and $x_j$ (left and right) lead to the same summed estimate at $x_s$. This shows a figure consisting of different types of lines. Elements of the figure described in the caption should be set in italics, in parentheses, as shown in this sample caption. The last sentence of a figure caption should generally end with a full stop, except when the caption is not a full sentence.
  • Figure 2: Centered, short example caption