Self-Parameterization Based Multi-Resolution Mesh Convolution Networks

TL;DR

This work introduces SPMM-Net, a self-parameterization based multi-resolution mesh convolution framework that extends image-style dense prediction to irregular 3D meshes. It constructs a multi-resolution mesh pyramid via bijective surface-to-surface mappings and employs area-aware pooling, barycentric upsampling, and face-convolution within a HRNet-inspired architecture to maintain high-resolution representations across stages. The approach achieves state-of-the-art performance on mesh segmentation and shape correspondence benchmarks, validating the effectiveness of preserving high-resolution information and cross-resolution fusion for dense mesh predictions. The work enables more accurate mesh analysis in geometric modeling while highlighting practical limitations related to mesh topology, parameterization quality, and certain CNN operations not supported.

Abstract

This paper addresses the challenges of designing mesh convolution neural networks for 3D mesh dense prediction. While deep learning has achieved remarkable success in image dense prediction tasks, directly applying or extending these methods to irregular graph data, such as 3D surface meshes, is nontrivial due to the non-uniform element distribution and irregular connectivity in surface meshes which make it difficult to adapt downsampling, upsampling, and convolution operations. In addition, commonly used multiresolution networks require repeated high-to-low and then low-to-high processes to boost the performance of recovering rich, high-resolution representations. To address these challenges, this paper proposes a self-parameterization-based multi-resolution convolution network that extends existing image dense prediction architectures to 3D meshes. The novelty of our approach lies in two key aspects. First, we construct a multi-resolution mesh pyramid directly from the high-resolution input data and propose area-aware mesh downsampling/upsampling operations that use sequential bijective inter-surface mappings between different mesh resolutions. The inter-surface mapping redefines the mesh, rather than reshaping it, which thus avoids introducing unnecessary errors. Second, we maintain the high-resolution representation in the multi-resolution convolution network, enabling multi-scale fusions to exchange information across parallel multi-resolution subnetworks, rather than through connections of high-to-low resolution subnetworks in series. These features differentiate our approach from most existing mesh convolution networks and enable more accurate mesh dense predictions, which is confirmed in experiments.

Figures (15)

• Figure 1: We construct a multi-resolution mesh pyramid from high-resolution input meshes using bijective surface-to-surface maps for any pair of adjacent meshes in the pyramid (left). Based on this, we design down-/up-sampling operators that enable us to easily adapt classic CNN dense prediction architectures and we construct a multi-resolution convolution network that maintains high-resolution representations throughout the network (middle). The resulting networks allow for both high-to-low and low-to-high feature aggregations, leading to high performance in mesh dense predictions (right).
• Figure 2: Generating the multi-resolution mesh pyramid from an input mesh. Bijective inter-surface maps between $M^{n+1}$ (blue) and $M^{n}$ (red) are visualizaed.
• Figure 3: We flatten the one-ring face-neighborhood of $e^{\star}$ using LSCM parameterization lscm2002. After $v^{\star}$ is generated by QEM in 3D space, we also flatten its one-ring face-neighborhood into the same UV domain.
• Figure 4: The process of recording and retrieving edge-to-edge intersection information during two consecutive iterations. From left to right: (a) A 3D patch is flattened and the edge $e^{\star}$ is to be collapsed. (b) The intersection points are computed in the UV domain and recorded as quadruplets, while the collapsed vertices are recorded as triplets. (c) In the second iteration, intersection points on the boundary edges of the blue patch are retrieved according to recorded quadruplets, and the triplets corresponding to the blue patch are also updated. (d) Intersection points for inner edges of the local patch are updated. (e) After several iterations, each face of the local patch contains a partial map of $M^{n+1}$.
• Figure 5: For input mesh (a), we visualize the decimated mesh with different values of weight $w$ for combining the approximation error and the distortion error. $w=0$ gives the pure QEM simplification that generates shape-preserving decimated mesh (b), while $w=0.1$ helps the reduction of thin triangles in the decimated result (c), and $w=0.3$ leads to the result with more uniform-areas (d).