Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ARF-Plus: Controlling Perceptual Factors in Artistic Radiance Fields for 3D Scene Stylization

Wenzhao Li, Tianhao Wu, Fangcheng Zhong, Cengiz Oztireli

TL;DR

ARF-Plus addresses the lack of perceptual controllability in radiance-field style transfer by introducing four controllable factors—color preservation, style scale, spatial targeting, and depth enhancement—within a reconstruction-and-optimization pipeline. It proposes tailored losses and a deferred back-propagation mechanism to apply these controls to neural radiance fields, including single- and multi-style scenarios. Quantitative and qualitative evaluations on real-world datasets demonstrate improved control fidelity (color, scale, spatial localization, depth) over the baseline ARF approach, with user studies supporting perceived usefulness. The work advances 3D scene stylization by enabling customized, multi-factor, and multi-style transfers with potential for practical applications and user-friendly tools.

Abstract

The radiance fields style transfer is an emerging field that has recently gained popularity as a means of 3D scene stylization, thanks to the outstanding performance of neural radiance fields in 3D reconstruction and view synthesis. We highlight a research gap in radiance fields style transfer, the lack of sufficient perceptual controllability, motivated by the existing concept in the 2D image style transfer. In this paper, we present ARF-Plus, a 3D neural style transfer framework offering manageable control over perceptual factors, to systematically explore the perceptual controllability in 3D scene stylization. Four distinct types of controls - color preservation control, (style pattern) scale control, spatial (selective stylization area) control, and depth enhancement control - are proposed and integrated into this framework. Results from real-world datasets, both quantitative and qualitative, show that the four types of controls in our ARF-Plus framework successfully accomplish their corresponding perceptual controls when stylizing 3D scenes. These techniques work well for individual style inputs as well as for the simultaneous application of multiple styles within a scene. This unlocks a realm of limitless possibilities, allowing customized modifications of stylization effects and flexible merging of the strengths of different styles, ultimately enabling the creation of novel and eye-catching stylistic effects on 3D scenes.

ARF-Plus: Controlling Perceptual Factors in Artistic Radiance Fields for 3D Scene Stylization

TL;DR

ARF-Plus addresses the lack of perceptual controllability in radiance-field style transfer by introducing four controllable factors—color preservation, style scale, spatial targeting, and depth enhancement—within a reconstruction-and-optimization pipeline. It proposes tailored losses and a deferred back-propagation mechanism to apply these controls to neural radiance fields, including single- and multi-style scenarios. Quantitative and qualitative evaluations on real-world datasets demonstrate improved control fidelity (color, scale, spatial localization, depth) over the baseline ARF approach, with user studies supporting perceived usefulness. The work advances 3D scene stylization by enabling customized, multi-factor, and multi-style transfers with potential for practical applications and user-friendly tools.

Abstract

The radiance fields style transfer is an emerging field that has recently gained popularity as a means of 3D scene stylization, thanks to the outstanding performance of neural radiance fields in 3D reconstruction and view synthesis. We highlight a research gap in radiance fields style transfer, the lack of sufficient perceptual controllability, motivated by the existing concept in the 2D image style transfer. In this paper, we present ARF-Plus, a 3D neural style transfer framework offering manageable control over perceptual factors, to systematically explore the perceptual controllability in 3D scene stylization. Four distinct types of controls - color preservation control, (style pattern) scale control, spatial (selective stylization area) control, and depth enhancement control - are proposed and integrated into this framework. Results from real-world datasets, both quantitative and qualitative, show that the four types of controls in our ARF-Plus framework successfully accomplish their corresponding perceptual controls when stylizing 3D scenes. These techniques work well for individual style inputs as well as for the simultaneous application of multiple styles within a scene. This unlocks a realm of limitless possibilities, allowing customized modifications of stylization effects and flexible merging of the strengths of different styles, ultimately enabling the creation of novel and eye-catching stylistic effects on 3D scenes.
Paper Structure (24 sections, 10 equations, 8 figures, 1 table)

This paper contains 24 sections, 10 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Methodology
  4. Color Preservation Control
  5. Scale Control
  6. Single Style Scale Control
  7. Multiple Style Scale Control
  8. Spatial Control
  9. Single Style Spatial Control
  10. Multiple Style Spatial Control
  11. Depth Enhancement Control
  12. Experiments and Evaluation
  13. Experiments Design
  14. Color Preservation Control
  15. Scale Control
  16. ...and 9 more sections

Figures (8)

  • Figure 1: An overview of ARF-Plus framework. It facilitates the control of various perceptual factors and permits the use of multiple style images as input, resulting in perceptual flexibility in 3D scene stylization.
  • Figure 2: Qualitative results of color preservation. Please refer to supplementary materials for better visualization.
  • Figure 3: Qualitative results of single style scale control. Our scale control effectively achieves style scaling. Please refer to supplementary materials for better visualization.
  • Figure 4: Qualitative results of multiple styles scale control. For better visualization, color is preserved by using our color preservation control. Our ARF-Plus with scale control can selectively scale up or scale down a particular style in stylization with blended multiple styles.
  • Figure 5: Qualitative results of spatial control with semantic segmentation masks: style transfer on the forward-facing scene - Room. Our ARF-Plus with spatial control effectively stylizes specific semantic objects - chair, table, and TV - within the scene.
  • ...and 3 more figures