Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geo-NVS-w: Geometry-Aware Novel View Synthesis In-the-Wild with an SDF Renderer

Anastasios Tsalakopoulos, Angelos Kanlis, Evangelos Chatzis, Antonis Karakottas, Dimitrios Zarpalas

TL;DR

Geo-NVS-w tackles in-the-wild novel view synthesis by grounding rendering in a high-fidelity Signed Distance Function (SDF) and pairing it with an octree feature volume for efficiency. The framework integrates an SDF-guided renderer, per-image appearance codes, and a Geometry-Preservation Loss (GPL) to preserve sharp geometry across views, achieving competitive quality while substantially reducing energy consumption. Key contributions include the dual-octree feature volume (foreground SDF-based and background NeRF-based), NeuS-style rendering, and a principled loss that protects geometric details from transient occluders. The approach yields photorealistic, geometrically coherent views in unconstrained datasets and offers favorable efficiency–quality trade-offs compared to prior in-the-wild NVS methods. This work highlights the enduring value of explicit geometric grounding for high-fidelity view synthesis in practical, unconstrained settings.

Abstract

We introduce Geo-NVS-w, a geometry-aware framework for high-fidelity novel view synthesis from unstructured, in-the-wild image collections. While existing in-the-wild methods already excel at novel view synthesis, they often lack geometric grounding on complex surfaces, sometimes producing results that contain inconsistencies. Geo-NVS-w addresses this limitation by leveraging an underlying geometric representation based on a Signed Distance Function (SDF) to guide the rendering process. This is complemented by a novel Geometry-Preservation Loss which ensures that fine structural details are preserved. Our framework achieves competitive rendering performance, while demonstrating a 4-5x reduction reduction in energy consumption compared to similar methods. We demonstrate that Geo-NVS-w is a robust method for in-the-wild NVS, yielding photorealistic results with sharp, geometrically coherent details.

Geo-NVS-w: Geometry-Aware Novel View Synthesis In-the-Wild with an SDF Renderer

TL;DR

Geo-NVS-w tackles in-the-wild novel view synthesis by grounding rendering in a high-fidelity Signed Distance Function (SDF) and pairing it with an octree feature volume for efficiency. The framework integrates an SDF-guided renderer, per-image appearance codes, and a Geometry-Preservation Loss (GPL) to preserve sharp geometry across views, achieving competitive quality while substantially reducing energy consumption. Key contributions include the dual-octree feature volume (foreground SDF-based and background NeRF-based), NeuS-style rendering, and a principled loss that protects geometric details from transient occluders. The approach yields photorealistic, geometrically coherent views in unconstrained datasets and offers favorable efficiency–quality trade-offs compared to prior in-the-wild NVS methods. This work highlights the enduring value of explicit geometric grounding for high-fidelity view synthesis in practical, unconstrained settings.

Abstract

We introduce Geo-NVS-w, a geometry-aware framework for high-fidelity novel view synthesis from unstructured, in-the-wild image collections. While existing in-the-wild methods already excel at novel view synthesis, they often lack geometric grounding on complex surfaces, sometimes producing results that contain inconsistencies. Geo-NVS-w addresses this limitation by leveraging an underlying geometric representation based on a Signed Distance Function (SDF) to guide the rendering process. This is complemented by a novel Geometry-Preservation Loss which ensures that fine structural details are preserved. Our framework achieves competitive rendering performance, while demonstrating a 4-5x reduction reduction in energy consumption compared to similar methods. We demonstrate that Geo-NVS-w is a robust method for in-the-wild NVS, yielding photorealistic results with sharp, geometrically coherent details.
Paper Structure (16 sections, 5 equations, 3 figures, 1 table)

This paper contains 16 sections, 5 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Novel View Synthesis
  4. Neural Surface Representation
  5. The Geo-NVS-w Framework
  6. Octree Feature Volume
  7. SDF-guided Volumetric Rendering
  8. Appearance and Transient Modules
  9. Geometry-Preservation Loss (GPL)
  10. Learning Objective
  11. Experiments
  12. Implementation Details
  13. Novel View Synthesis Benchmarks
  14. Energy Analysis
  15. Conclusion
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Overview of the Geo-NVS-w Framework. For a given camera ray, we march through an octree containing feature grids for the foreground (SDF-based) and background (NeRF-based). Within the foreground unit sphere, interpolated features are passed to MLPs to predict an SDF value $s(\mathbf{x})$ and a color $c(\mathbf{x}, \mathbf{d})$. We use the NeuS rendering formula to convert SDF values into alpha-compositing weights $w_i$, ensuring rendered colors are tightly coupled to the underlying surface. These weights are used to accumulate color, which is then composed with the background. Our Geometry-Preservation Loss (GPL) ensures the transient mask does not erode high-curvature details.
  • Figure 2: Qualitative results on Phototourism scenes. Top to bottom: Sacré-Cœur, Trevi Fountain, Brandenburg Gate. (a) Input image. (b) Rendered result with Geo-NVS-w from the same viewpoint and appearance embedding. (c) Estimated transiency mask. (d) Visualization of the geometry-preservation map (accumulated SDF gradients along each ray).
  • Figure 3: Energy vs. quality trade-off. Geo-NVS-w achieves high PSNR with significantly less training time and cumulative energy consumption (kWh) compared to baseline NeRF-W, making it a more efficient and scalable solution.