A Latent Implicit 3D Shape Model for Multiple Levels of Detail

TL;DR

This work tackles the limitation of implicit neural shape representations that support only a single level of detail by introducing a latent-conditioned, band-limited multiscale network that jointly learns per-layer frequencies and phases. A design choice that concatenates the latent code to hidden activations preserves the network’s band-limiting while enabling a single latent vector to parameterize multiple shapes across LoDs, and enables fast, coarse-to-fine iso-surface extraction. Experiments on ShapeNet cars, chairs, and airplanes show that the approach yields smoother surfaces at low LoDs and competitive geometric accuracy at high LoDs, with a clear speed-accuracy trade-off and interactive latent-space exploration. The method also supports shape completion from partial supervision and includes thorough ablations validating the importance of learning coordinate embeddings and the latent conditioning strategy.

Abstract

Implicit neural representations map a shape-specific latent code and a 3D coordinate to its corresponding signed distance (SDF) value. However, this approach only offers a single level of detail. Emulating low levels of detail can be achieved with shallow networks, but the generated shapes are typically not smooth. Alternatively, some network designs offer multiple levels of detail, but are limited to overfitting a single object. To address this, we propose a new shape modeling approach, which enables multiple levels of detail and guarantees a smooth surface at each level. At the core, we introduce a novel latent conditioning for a multiscale and bandwith-limited neural architecture. This results in a deep parameterization of multiple shapes, where early layers quickly output approximated SDF values. This allows to balance speed and accuracy within a single network and enhance the efficiency of implicit scene rendering. We demonstrate that by limiting the bandwidth of the network, we can maintain smooth surfaces across all levels of detail. At finer levels, reconstruction quality is on par with the state of the art models, which are limited to a single level of detail.

Figures (16)

• Figure 1: Network Architecture. coordinate $\mathbf{x}$ is encoded with parametric functions $\mathbf{g}_i(\cdot)$, and concatenated with the latent code $\mathbf{l}$. Linear layers $(\mathbf{W}_i, \mathbf{b}_i)$ and hadamard products yield the intermediate activations $\mathbf{z}_i$. At level $i$, the SDF output is predicted with the output layer $(\mathbf{W}_i^{out}, \mathbf{b}_i^{out})$. All network parameters ${\Omega = \{\mathbf{W}_i, \mathbf{b}_i, \mathbf{W}_i^{out}, \mathbf{b}_i^{out}, \overline{\boldsymbol{\omega}_i}, \boldsymbol{\phi}_i \}}$ are optimized jointly with the latent codebook.
• Figure 2: Architecture Design. When concatenating a latent code to the input coordinates (Design 1) or the output layers (Design 2), the resulting shapes are unsatisfactory. When the latent code is concatenated to the hidden layers (Design 3 (Ours)), the generated shapes are significantly closer to the ground truth.
• Figure 3: Test Shape Reconstructions with varying levels of detail. The lowest possible level of detail ($i=1$) is smooth and already captures a very coarse structure of the shape. As $i$ increases, more details appear (up to saturation).
• Figure 4: Linear Interpolation in the latent space translates into smooth shape deformations. Smaller objects correspond to a lower level of detail ($i=1$), and follow a similar transformation to the more detailed ones ($i=12$).
• Figure 5: Surface Regularity (SR): to measure regularity we average the vertex displacements that one step of Laplacian smoothing would cause. As shown in the insets, this captures surface cracks and bumps (left: 1 layer ReLU-net, right: ours at $i=1$).