Neural Geometry Image-Based Representations with Optimal Transport (OT)

TL;DR

The paper tackles efficient storage and restoration of dense 3D meshes by introducing a decoder-free neural geometry-image representation. It converts irregular meshes to regular geometry-image mipmaps using an Optimal Transport–based area-preserving parameterization, enabling single-pass reconstruction of high-quality meshes. A CNN operates directly in geometry-image space to restore full-resolution images from low-resolution mipmaps, yielding continuous levels of detail on GPUs without progressive decoders. Experiments on Thingi10K demonstrate state-of-the-art storage efficiency and reconstruction accuracy (CR, CD, HD) while enabling decoder-free, GPU-friendly LoD control, outperforming neural overfitting and subdivision baselines.

Abstract

Neural representations for 3D meshes are emerging as an effective solution for compact storage and efficient processing. Existing methods often rely on neural overfitting, where a coarse mesh is stored and progressively refined through multiple decoder networks. While this can restore high-quality surfaces, it is computationally expensive due to successive decoding passes and the irregular structure of mesh data. In contrast, images have a regular structure that enables powerful super-resolution and restoration frameworks, but applying these advantages to meshes is difficult because their irregular connectivity demands complex encoder-decoder architectures. Our key insight is that a geometry image-based representation transforms irregular meshes into a regular image grid, making efficient image-based neural processing directly applicable. Building on this idea, we introduce our neural geometry image-based representation, which is decoder-free, storage-efficient, and naturally suited for neural processing. It stores a low-resolution geometry-image mipmap of the surface, from which high-quality meshes are restored in a single forward pass. To construct geometry images, we leverage Optimal Transport (OT), which resolves oversampling in flat regions and undersampling in feature-rich regions, and enables continuous levels of detail (LoD) through geometry-image mipmapping. Experimental results demonstrate state-of-the-art storage efficiency and restoration accuracy, measured by compression ratio (CR), Chamfer distance (CD), and Hausdorff distance (HD).

Figures (8)

• Figure 1: Continuous Levels of Detail. Our neural geometry image–based representation with Optimal Transport (OT) restores full-resolution geometry images at Level 0 from a low-resolution input (Level 7) in a single forward pass. By leveraging OT, we avoid oversampling in flat regions and undersampling of facial details, a property we call uniform sampling. Both rows show 3D meshes reconstructed from geometry image mipmaps across different levels. This demonstrates continuous levels of detail without the need for decoders, exploiting GPU-optimized image mipmapping and achieving state-of-the-art storage efficiency measured by Compression Ratio (CR) and reconstruction accuracy measured by Chamfer Distance (CD) and Hausdorff Distance (HD).
• Figure 2: Overview of OT for geometry images. A 3D surface mesh (a) is flattened into the 2D domain by conformal initialization (b), preserving angles. Optimal Transport refinement (c) applies a measure-preserving map that redistributes area for balanced sampling. The final OT-based geometry image (d) provides a compact, area-preserving representation of the 3D surface mesh for efficient storage and neural processing.
• Figure 3: Neural Geometry Image-Based Representation with Optimal Transport (OT). Our pipeline is decoder-free, memory-efficient, and enables continuous geometric Level-of-Detail (LoD) entirely on the GPU without progressive refinement. (1) Geometry images are first constructed using Optimal Mass Transport (OMT) to preserve surface area and regularize sampling across irregular meshes. (2) A geometry image super-resolution network directly upsamples a low-resolution mipmapped geometry image (position and normal maps) in a single forward pass, avoiding the need for multiple decoders. (3) The reconstructed geometry image is hierarchically mipmapped, yielding a pyramid of geometry images that allows continuous LoD control. At runtime, the GPU can seamlessly select and render any desired resolution, balancing storage and rendering constraints while preserving both global shape and local detail.
• Figure 4: Projection from 2D image space to 3D surface. The 2D OT-based geometry image is first triangulated in image space and then projected back to reconstruct the 3D surface mesh.
• Figure 5: Qualitative comparison with state-of-the-art methods under imbalanced sampling during mesh triangulation and extraction. Numbers below each method denote Chamfer Distance (CD, $\times 10^{-4}$) / Hausdorff Distance (HD, $\times 10^{-2}$), where lower values indicate better reconstruction quality. Our method achieves better preservation of surface details and greater robustness to sampling imbalance by applying OT to construct strictly area-preserving geometry images. Each 3D surface mesh is extracted from its corresponding pair of position and normal geometry images shown below it.