NESI: Shape Representation via Neural Explicit Surface Intersection

TL;DR

This work proposes a novel learned alternative to learned implicit or parametric representations, NESI, based on intersections of localized explicit, or height-field, surfaces that directly supports a wider range of processing operations than implicit alternatives, including occupancy queries and parametric access.

Abstract

Compressed representations of 3D shapes that are compact, accurate, and can be processed efficiently directly in compressed form, are extremely useful for digital media applications. Recent approaches in this space focus on learned implicit or parametric representations. While implicits are well suited for tasks such as in-out queries, they lack natural 2D parameterization, complicating tasks such as texture or normal mapping. Conversely, parametric representations support the latter tasks but are ill-suited for occupancy queries. We propose a novel learned alternative to these approaches, based on intersections of localized explicit, or height-field, surfaces. Since explicits can be trivially expressed both implicitly and parametrically, NESI directly supports a wider range of processing operations than implicit alternatives, including occupancy queries and parametric access. We represent input shapes using a collection of differently oriented height-field bounded half-spaces combined using volumetric Boolean intersections. We first tightly bound each input using a pair of oppositely oriented height-fields, forming a Double Height-Field (DHF) Hull. We refine this hull by intersecting it with additional localized height-fields (HFs) that capture surface regions in its interior. We minimize the number of HFs necessary to accurately capture each input and compactly encode both the DHF hull and the local HFs as neural functions defined over subdomains of R^2. This reduced dimensionality encoding delivers high-quality compact approximations. Given similar parameter count, or storage capacity, NESI significantly reduces approximation error compared to the state of the art, especially at lower parameter counts.

Figures (27)

• Figure 1: A shape (a) represented (d) as the intersection (green) of a DHF hull (b) and an additional HF (c). Localizing the HF to a narrower parameter domain $\Omega$ (e), by implicitly assuming it to match the DHF elsewhere, reduces representation redundancy.
• Figure 2: NESI approximations (e) of complex inputs (a) are much more detailed and accurate than those generated by leading implicit alternatives: (b) vqad, (c) SIREN siren, and (d) nglod, despite using fewer parameters.
• Figure 3: Depth fusion methods (a,c) use intersections of multiple depth maps with fixed, input independent axis directions (visualized by arrows) to approximate input shapes. These methods often fail to approximate large parts of the input surfaces (red in a and c) not visible along these axes. (a) 20 axis directions evenly distributed on a circle in the $x-y$ plane shade1998layered; (c) 6 directions aligned with the +/- axes of the object's coordinate system Richter2018Matryoshka. (b,d) ESI accurately captures both shapes using a single DHF with automatically computed optimal axis.
• Figure 4: Given an input 3D shape (a), we compute a DHF hull and HFs whose intersection accurately approximates the input (b). We then employ MLPs (c) to encode the DHF and HFs as $R^2 \rightarrow R$ functions (on the HF only purple areas are inside the $\tilde{\Omega}$ parameter domain) (d). At inference time we combine (intersect) the MLP outputs (e).
• Figure 5: Approximation criteria: given the inputs on the left (a), using only surface coverage as an axis selection criterion produces VE intersections that may contain extra undesirable connected components (b, highlighted in red). Optimizing for both volumetric approximation and surface coverage produces the outputs we seek (c).