Thinking Like Van Gogh: Structure-Aware Style Transfer via Flow-Guided 3D Gaussian Splatting

TL;DR

The paper addresses the gap in 3D style transfer where existing methods preserve photographic topology and treat style as texture or color statistics. It introduces a mesh-free, flow-guided geometric advection framework for 3D Gaussian Splatting that back-propagates 2D painting flow into 3D primitives to form flow-aligned brushstrokes, explicitly exaggerating geometry while suppressing photorealistic detail. Key contributions include a projection-based flow guidance mechanism with a flow-alignment energy $\mathcal{L}_{\text{align}}^{(i)}$, a luminance-structure decoupling strategy that isolates geometric deformation from color optimization via $\mathcal{L}_{style}$ in the luminance channel and color statistics in Lab space, and a VLM-as-a-Judge evaluation framework for artistic authenticity. The results demonstrate coherent, impasto-like 3D brushwork that follows scene topology across views, validated by qualitative comparisons, a semantic evaluation using large multimodal models, and user studies, highlighting practical impact for non-photorealistic rendering and digital art synthesis in 3D.

Abstract

In 1888, Vincent van Gogh wrote, "I am seeking exaggeration in the essential." This principle, amplifying structural form while suppressing photographic detail, lies at the core of Post-Impressionist art. However, most existing 3D style transfer methods invert this philosophy, treating geometry as a rigid substrate for surface-level texture projection. To authentically reproduce Post-Impressionist stylization, geometric abstraction must be embraced as the primary vehicle of expression. We propose a flow-guided geometric advection framework for 3D Gaussian Splatting (3DGS) that operationalizes this principle in a mesh-free setting. Our method extracts directional flow fields from 2D paintings and back-propagates them into 3D space, rectifying Gaussian primitives to form flow-aligned brushstrokes that conform to scene topology without relying on explicit mesh priors. This enables expressive structural deformation driven directly by painterly motion rather than photometric constraints. Our contributions are threefold: (1) a projection-based, mesh-free flow guidance mechanism that transfers 2D artistic motion into 3D Gaussian geometry; (2) a luminance-structure decoupling strategy that isolates geometric deformation from color optimization, mitigating artifacts during aggressive structural abstraction; and (3) a VLM-as-a-Judge evaluation framework that assesses artistic authenticity through aesthetic judgment instead of conventional pixel-level metrics, explicitly addressing the subjective nature of artistic stylization.

Figures (8)

• Figure 1: Subjectivity over Physics. While the baseline method (middle) rigidly preserves photorealistic perspective and lighting—treating style as a flat texture—our method (right) prioritizes subjective geometric flow. We demonstrate that authentic stylization requires sacrificing objective physical fidelity to reconstruct the expressive structural abstraction of the artist.
• Figure 2: Directional Syntax. Van Gogh (left): turbulent flow. Munch (right): laminar flow. Both prioritize geometric coherence.
• Figure 3: The Analogy between Artistic Cognition and Computational Simulation. Top: Artistic process—perceiving 3D reality, deciding stroke orientations, creating 2D expression. Bottom: Our computational pipeline mirrors this—extracting flow from style, modeling as Gaussians, rendering via geometric advection.
• Figure 4: The Projection-Induced Advection Process.
• Figure 5: Overview of the Thinking Like Van Gogh Framework.