Inverse Rendering for High-Genus Surface Meshes from Multi-View Images

TL;DR

The paper tackles inverse rendering of high-genus surfaces from multi-view images using a mesh-based representation. It introduces a topology-informed optimization pipeline that couples adaptive V-cycle remeshing with a re-parametrized Adam optimizer and enforces topology via Gauss-Bonnet–based genus constraints, ensuring genus invariance with $g = 1 - \chi(S)/2$ and $\chi(S) = |V|+|F|-|E|$. Key contributions include topology-aware remeshing to preserve topological features, explicit genus matching, and empirical improvements in Chamfer Distance and Volume IoU, especially for high-genus geometries. The approach demonstrates robust reconstruction quality suitable for downstream tasks like relighting, simulation, and 3D printing.

Abstract

We present a topology-informed inverse rendering approach for reconstructing high-genus surface meshes from multi-view images. Compared to 3D representations like voxels and point clouds, mesh-based representations are preferred as they enable the application of differential geometry theory and are optimized for modern graphics pipelines. However, existing inverse rendering methods often fail catastrophically on high-genus surfaces, leading to the loss of key topological features, and tend to oversmooth low-genus surfaces, resulting in the loss of surface details. This failure stems from their overreliance on Adam-based optimizers, which can lead to vanishing and exploding gradients. To overcome these challenges, we introduce an adaptive V-cycle remeshing scheme in conjunction with a re-parametrized Adam optimizer to enhance topological and geometric awareness. By periodically coarsening and refining the deforming mesh, our method informs mesh vertices of their current topology and geometry before optimization, mitigating gradient issues while preserving essential topological features. Additionally, we enforce topological consistency by constructing topological primitives with genus numbers that match those of ground truth using Gauss-Bonnet theorem. Experimental results demonstrate that our inverse rendering approach outperforms the current state-of-the-art method, achieving significant improvements in Chamfer Distance and Volume IoU, particularly for high-genus surfaces, while also enhancing surface details for low-genus surfaces.

Figures (8)

• Figure 1: Inverse Rendering for High-Genus Surface Meshes from Multi-View Images. (Top) Reconstructions using the SOTA method Nicolet2021Large, which produces incorrect genus number, leading to incorrect topology. (Middle) Our method with the correct genus number, leading to correct topology. (Bottom) Challenging High-Genus Ground Truth. Please see Appendix D.3 for quantitative results.
• Figure 2: Topological Consistency: The top row shows a sphere with genus 0, not homeomorphic to the Bob surface's genus 1 ground truth, resulting in topological inconsistency. The bottom row shows a torus with genus 1, ensuring topological consistency.
• Figure 3: Overall Pipeline: A triangulated topological primitive, with genus matching that of the ground truth, undergoes adaptive V-cycle remeshing with periodic coarsening and refining stages, followed by optimization using an Adam-based optimizer to minimize the multiview rendering loss.
• Figure 4: Visualization of topology-preserving local mesh operationsmeshFigs23. (a) Edge collapse, (b) Edge split, and (c) Edge flip.
• Figure 5: Qualitative High-Genus Reconstruction: Rendered Views in Normal Maps Using Topologically Consistent Triangulated Primitives (Genus 1, 2, 3, 4 and 5). Please see Appendix A.2 for the complete set of multi-view high-genus surface qualitative results.