NeuSEditor: From Multi-View Images to Text-Guided Neural Surface Edits

TL;DR

NeuSEditor tackles the challenge of editing neural implicit surfaces from multi-view images by introducing an identity-preserving, two-stage framework. The method separates scene identity (foreground and background) from edits and learns a target renderer guided by geometry-aware distillation that incorporates diffusion priors and Phong shading to improve geometric fidelity. Key contributions include an identity-focused architecture, a geometry-aware distillation loss, and explicit separation of background editing to prevent unwanted changes to foreground features. Empirical results on DTU, Blender, and IN2N datasets show superior rendering fidelity, reduced artifacts, and better preservation of scene identity compared with state-of-the-art baselines, with robust performance in both objective metrics and user studies. This approach advances robust, text-guided 3D editing for implicit surfaces, enabling precise, controllable edits without continual dataset updates.

Abstract

Implicit surface representations are valued for their compactness and continuity, but they pose significant challenges for editing. Despite recent advancements, existing methods often fail to preserve identity and maintain geometric consistency during editing. To address these challenges, we present NeuSEditor, a novel method for text-guided editing of neural implicit surfaces derived from multi-view images. NeuSEditor introduces an identity-preserving architecture that efficiently separates scenes into foreground and background, enabling precise modifications without altering the scene-specific elements. Our geometry-aware distillation loss significantly enhances rendering and geometric quality. Our method simplifies the editing workflow by eliminating the need for continuous dataset updates and source prompting. NeuSEditor outperforms recent state-of-the-art methods like PDS and InstructNeRF2NeRF, delivering superior quantitative and qualitative results. For more visual results, visit: neuseditor.github.io.

Figures (12)

• Figure 1: NeuSEditor is a novel method for text-guided neural surface editing from multi-view calibrated images. It generates an edited scene represented as a neural implicit field. Here, the sample inputs on the left are used to generate various 3D textured meshes.
• Figure 2: Our network architecture retains information about the identity (learned source) of the original scene, including background and foreground details, while simultaneously learning the edited (target) render and geometry. Text prompt: turn it to a polar bear.
• Figure 3: Overview of our two stage scene editing pipeline. In the first stage (source learning) the network captures the scene's identity, including its geometry and appearance, using SDF based volume rendering with progressive hash encodings. In the second stage (target learning) a target renderer, conditioned on the source renderer and guided by a geometry aware distillation loss, edits the scene. Here, $I_{\mathrm{src}}$ and $I_{\mathrm{tgt}}$ denote the source and target renderings respectively, while $P_{\mathrm{tgt}}$ represents Phong shading. Within the pipeline, the red arrows highlights the transfer of knowledge between source and target, the integration of Phong shading, and its role in the loss formulation.
• Figure 4: Influence of hyperparameter guidance scale. Left: two sample input images from the DTU dataset (scan105 scene) and the Blender dataset (hotdog scene), respectively. Right: the edited mesh renderings of the corresponding scenes at guidance scales from 100 to 350. The text prompt for the top row is 'add suit', while the prompt for the bottom row is 'make it bananas'.
• Figure 5: The left column displays an input view from the DTU dataset scan24, while the right column shows the foreground render, depth map, and normal map. The top row presents results from the naive Hash Grids + NeuS + PDS loss approach. The middle row shows our architecture with PDS loss, and the bottom row shows our architecture with Phong Enhanced PDS loss ($\lambda_\text{PDS}=1.$ and $\lambda_\text{PE} = 0.2$). The text prompt is 'make it a church'.