$α$Surf: Implicit Surface Reconstruction for Semi-Transparent and Thin Objects with Decoupled Geometry and Opacity

TL;DR

This work presents $\alpha$Surf, a novel surface representation with decoupled geometry and opacity for the reconstruction of semi-transparent and thin surfaces where the colors mix, achieving better reconstruction quality than state-of-the-art SDF and NeRF methods.

Abstract

Implicit surface representations such as the signed distance function (SDF) have emerged as a promising approach for image-based surface reconstruction. However, existing optimization methods assume solid surfaces and are therefore unable to properly reconstruct semi-transparent surfaces and thin structures, which also exhibit low opacity due to the blending effect with the background. While neural radiance field (NeRF) based methods can model semi-transparency and achieve photo-realistic quality in synthesized novel views, their volumetric geometry representation tightly couples geometry and opacity, and therefore cannot be easily converted into surfaces without introducing artifacts. We present $α$Surf, a novel surface representation with decoupled geometry and opacity for the reconstruction of semi-transparent and thin surfaces where the colors mix. Ray-surface intersections on our representation can be found in closed-form via analytical solutions of cubic polynomials, avoiding Monte-Carlo sampling and is fully differentiable by construction. Our qualitative and quantitative evaluations show that our approach can accurately reconstruct surfaces with semi-transparent and thin parts with fewer artifacts, achieving better reconstruction quality than state-of-the-art SDF and NeRF methods. Website: https://alphasurf.netlify.app/

Figures (18)

• Figure 1: Illustration. We illustrate the representations of NeuS, NeRF, and our method, as well as reconstructed surfaces. NeuS neus uses SDF to optimize for opaque surfaces and hence misses translucent or thin surfaces with blending effects in the reconstruction. NeRF methods such as Plenoxels plenoxels can represent semi-transparency with density field, but as density couples both occupancy and opacity, surfaces extracted from it would contain holes or redundant surface floater. In contrast, our approach models decoupled surface and opacity fields. We use a surface field without Eikonal constraint and multiple level sets $\tau_0,\tau_1,...$ to model geometry with different levels of confidence and opacity, and utilize a closed-form intersection formula to enable differentiable rendering, and hence can accurately reconstruct surfaces exhibiting semi-transparency.
• Figure 1: Chamfer distance $\downarrow \times 10^{-2}$ on synthetic datasets. We highlight the best methods. We justify the choice of density level set values for Plenoxels and MipNeRF360 in the supplementary.
• Figure 2: Blending effect. a) Real-world capture takes incoming light from multiple rays per pixel. Pixels that are partially occupied by an opaque object are therefore rendered as a mixture of the object and background color. b) With one-sample rendering in reconstruction, it becomes necessary for extremely tiny objects to be modeled as semi-transparent for the pixel color to match the ground truth. c), d) Our representation is fully capable of representing this phenomenon, leading to the accurate surface reconstruction of thin structures.
• Figure 3: Method. a) Our surface representation is based on a voxel grid storing explicit values, without neural networks; see Section \ref{['sec:representation']}. b) We utilize a closed-form and differentiable method to compute ray-surface intersection. This is achieved by solving a cubic polynomial of depth $t$ with known parameters $f_0,...,f_3$ and $\tau_i$; see Section \ref{['sec:rendering']}. c) We incorporate surface-specific regularization such as truncated alpha compositing to obtain clean and accurate surfaces; see Section \ref{['sec:optimization']}. d) We utilize a coarse initialization via Plenoxels plenoxels to start with roughly correct yet noisy surfaces. e) The optimization results in clean and complete surfaces in the end.
• Figure 4: Surface reconstruction with lower resolutions. Our method can reconstruct thin surfaces under a low resolution, where thin structures such as ropes heavily blend with the background.