CoFie: Learning Compact Neural Surface Representations with Coordinate Fields

TL;DR

CoFie introduces Coordinate Fields to disentangle transformation information from local geometry in neural surface representations, enabling a compact, voxel-based plus implicit decoding of local SDFs. By applying explicit per-voxel frames and a 5-layer MLP with a single quadratic top layer, CoFie achieves strong generalization to novel shapes while using substantially fewer parameters than prior methods. The theoretical analysis shows local SDFs are nonlinear and benefit from quadratic components, justifying the design, and experiments demonstrate significant accuracy gains over generalizable baselines and competitive performance on real scans. This work advances compact, generalizable neural surface representations for 3D reconstruction with potential impact on shape modeling and digital content creation.

Abstract

This paper introduces CoFie, a novel local geometry-aware neural surface representation. CoFie is motivated by the theoretical analysis of local SDFs with quadratic approximation. We find that local shapes are highly compressive in an aligned coordinate frame defined by the normal and tangent directions of local shapes. Accordingly, we introduce Coordinate Field, which is a composition of coordinate frames of all local shapes. The Coordinate Field is optimizable and is used to transform the local shapes from the world coordinate frame to the aligned shape coordinate frame. It largely reduces the complexity of local shapes and benefits the learning of MLP-based implicit representations. Moreover, we introduce quadratic layers into the MLP to enhance expressiveness concerning local shape geometry. CoFie is a generalizable surface representation. It is trained on a curated set of 3D shapes and works on novel shape instances during testing. When using the same amount of parameters with prior works, CoFie reduces the shape error by 48% and 56% on novel instances of both training and unseen shape categories. Moreover, CoFie demonstrates comparable performance to prior works when using only 70% fewer parameters.

Figures (11)

• Figure 1: CoFie is a local geometry-aware shape representation. (Left) CoFie divides a shape into non-overlapping local patches, where each local patch is represented by an MLP-based Signed Distance Function. (Right) CoFie introduces Coordinate Field, which attaches a coordinate frame to each local patch. It transforms local patches from the world coordinate system to an aligned coordinate system, reducing shape complexity.
• Figure 2: Overview of CoFie. CoFie represents a shape using a hybrid representation of voxels/cells and local implicit functions. (Left) For preparing the data for training the MLP-based local implicit functions, we split the training shapes into local shapes and initialize their coordinate frames using PCA. (Right) During training, a point will be transformed to the aligned coordinate of all local shapes using the coordinate frame. The MLP takes the transformed point and the latent code of the local shape to predict its SDF value. During testing, we fix the MLP, optimizing the latent codes and coordinate fields of valid cells.
• Figure 3: Diveristy and quality of meshes that CoFie can represent. The results include both novel instances from ShapeNet training categories (top left), instances from ShapeNet unseen categories (bottom left), and real shapes from the Thingi dataset (right). We visualize the shapes with surface normal to better show their geometry. Please see the appendix for comparisons with ground-truth.
• Figure 4: Shape errors on novel instances of the ShapeNet training categories. We report chamfer distance ($10^{-4}$) and highlight the best.
• Figure 5: Trade-off between accuracy and model size ( notified by the radius of circles).

Theorems & Definitions (3)

• Proposition 1
• Proposition 2
• Proposition 3