AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation

TL;DR

AtlasNet introduces a surface-based generative framework that represents 3D shapes as a union of learnable 2D-to-3D charts (an atlas) mapped from unit squares. By jointly learning the chart parameterizations and a shape embedding, it enables high-resolution mesh generation and UV parameterization, and supports arbitrary sampling without memory blow-up. The method demonstrates strong performance on ShapeNet for auto-encoding and single-view reconstruction, outperforming several point- and voxel-based baselines and enabling applications such as interpolation, correspondences, and texture mapping. This surface-centric approach offers a scalable path toward high-fidelity 3D meshes with practical texture and meshing capabilities.

Abstract

We introduce a method for learning to generate the surface of 3D shapes. Our approach represents a 3D shape as a collection of parametric surface elements and, in contrast to methods generating voxel grids or point clouds, naturally infers a surface representation of the shape. Beyond its novelty, our new shape generation framework, AtlasNet, comes with significant advantages, such as improved precision and generalization capabilities, and the possibility to generate a shape of arbitrary resolution without memory issues. We demonstrate these benefits and compare to strong baselines on the ShapeNet benchmark for two applications: (i) auto-encoding shapes, and (ii) single-view reconstruction from a still image. We also provide results showing its potential for other applications, such as morphing, parametrization, super-resolution, matching, and co-segmentation.

Figures (14)

• Figure 1: Given input as either a 2D image or a 3D point cloud (a), we automatically generate a corresponding 3D mesh (b) and its atlas parameterization (c). We can use the recovered mesh and atlas to apply texture to the output shape (d) as well as 3D print the results (e).
• Figure 2: Shape generation approaches. All methods take as input a latent shape representation (that can be learned jointly with a reconstruction objective) and generate as output a set of points. (a) A baseline deep architecture would simply decode this latent representation into a set of points of a given size. (b) Our approach takes as additional input a 2D point sampled uniformly in the unit square and uses it to generate a single point on the surface. Our output is thus the continuous image of a planar surface. In particular, we can easily infer a mesh of arbitrary resolution on the generated surface elements. (c) This strategy can be repeated multiple times to represent a 3D shape as the union of several surface elements.
• Figure 3: Auto-encoder. We compare the original meshes (a) to meshes obtained by running PSR on the point clouds generated by the baseline (b) and on the densely sampled point cloud from our generated mesh (c), and to our method generating a surface from a sphere (d), 1 (e), 5 (f), 25 (g), and 125(h) learnable parameterizations. Notice the fine details in (g) and (h) : e.g. the plane's engine and the jib of the ship.
• Figure 4: Generalization. (a) Our method (25 patches) can generate surfaces close to a category never seen during training. It, however, has more artifacts than if it has seen the category during training (b), e.g., thin legs and armrests.
• Figure 5: Single-view reconstruction comparison. From a 2D RGB image (a), 3D-R2N2 choy20163d reconstructs a voxel-based 3D model (b), HSP Hane:2017 reconstructs a octree-based 3D model (c), PointSetGen Fan:2017:cvpr a point cloud based 3D model (d), and our AtlasNet a triangular mesh (e).

Theorems & Definitions (4)

• Proposition 1
• proof
• Proposition 2
• proof