Editing Implicit and Explicit Representations of Radiance Fields: A Survey

TL;DR

The paper surveys

Abstract

Neural Radiance Fields (NeRF) revolutionized novel view synthesis in recent years by offering a new volumetric representation, which is compact and provides high-quality image rendering. However, the methods to edit those radiance fields developed slower than the many improvements to other aspects of NeRF. With the recent development of alternative radiance field-based representations inspired by NeRF as well as the worldwide rise in popularity of text-to-image models, many new opportunities and strategies have emerged to provide radiance field editing. In this paper, we deliver a comprehensive survey of the different editing methods present in the literature for NeRF and other similar radiance field representations. We propose a new taxonomy for classifying existing works based on their editing methodologies, review pioneering models, reflect on current and potential new applications of radiance field editing, and compare state-of-the-art approaches in terms of editing options and performance.

Figures (7)

• Figure 1: Transferring mesh deformation to an implicit radiance field causing ray bending (NeRFDeformer NeRFDeformer)
• Figure 2: Conditional NeRF on shape and appearance code graf. The appearance code $z_a$ only affects appearance and not density.
• Figure 3: Optimization pipeline of ARF arf and its NNFM loss.
• Figure 4: 3D generative pipeline proposed by DreamFusion dreamfusion.DreamFusion diffuses the rendered images and reconstructs it with a conditional Imagen to predict the injected noise $\hat{\epsilon}_\phi(z_t|y;t)$. The injected noise is then subtracted from and backpropagated to the NeRF.
• Figure 5: Unified compositional editing of NeRFs wang2023learning.