CageNet: A Meta-Framework for Learning on Wild Meshes

TL;DR

CageNet presents a meta-framework that enables learning on wild, multi-component, and non-manifold meshes by enveloping the input with a fully automatic, single-component cage and learning on the cage with a differentiable mapping to the original mesh. By employing generalized barycentric coordinates to transfer cage-based signals back to the input geometry, CageNet leverages robust mesh networks (e.g., DiffusionNet SharpACO22) to perform segmentation and skinning weight prediction with strong generalization to wild meshes. The approach achieves parity with state-of-the-art on clean meshes while outperforming baselines on wild data, and it demonstrates competitive skinning performance against specialized methods, aided by cage-offset augmentation and thoughtful coordinate choices. This framework broadens the applicability of generic mesh networks to real-world, imperfect geometry, enabling scalable learning on diverse mesh corpora and tasks with practical impact for 3D shape analysis and animation.

Abstract

Learning on triangle meshes has recently proven to be instrumental to a myriad of tasks, from shape classification, to segmentation, to deformation and animation, to mention just a few. While some of these applications are tackled through neural network architectures which are tailored to the application at hand, many others use generic frameworks for triangle meshes where the only customization required is the modification of the input features and the loss function. Our goal in this paper is to broaden the applicability of these generic frameworks to "wild", i.e. meshes in-the-wild which often have multiple components, non-manifold elements, disrupted connectivity, or a combination of these. We propose a configurable meta-framework based on the concept of caged geometry: Given a mesh, a cage is a single component manifold triangle mesh that envelopes it closely. Generalized barycentric coordinates map between functions on the cage, and functions on the mesh, allowing us to learn and test on a variety of data, in different applications. We demonstrate this concept by learning segmentation and skinning weights on difficult data, achieving better performance to state of the art techniques on wild meshes.

Figures (12)

• Figure 1: When the input is a single component manifold mesh (left), the existing method SharpACO22 and our CageNet lead to comparable results. However, when the input contains multiple connected components (right), our method results in better generalization. We show the computed segmentation color coded on the mesh, and the resulting accuracy.
• Figure 2: Given an input mesh $M$, possibly with multiple components and non-manifold elements, our CageNet constructs a single component, manifold cage $\widetilde{M}$ and the corresponding mapping operator $\mathcal{P}$ which maps signals from $\widetilde{M}$ to $M$. We then apply a neural network for "nice" meshes $\mathcal{F}_\theta$ on the cage $\widetilde{M}$ (where the input features $\tilde{x}$ are computed). The learned output features $\tilde{y}$ of the cage are projected using $\mathcal{P}$ on the input mesh, yielding the output features $y$ on $M$, which are used to compute the loss on $M$, where the training data is provided.
• Figure 3: Our method is robust to different cage resolutions, an advantage inherited from DiffusionNet SharpACO22. We show that the segmentation results are comparable (bottom row) even if the cages have different resolutions (top row). We show the accuracy of the segmentation result compared to the ground truth. The three leftmost cages are generated using our approach, whereas the rightmost cage is generated by guo2023robust.
• Figure 4: The choice of coordinates influences the generalization of CageNet. When the input is a single manifold mesh without interior structure (top row), CageNet can overfit the segmentation perfectly using all three coordinate choices. But when the input contains interior components (bottom rows), the harmonic coordinates fail near the brain region (second column), while the biharmonic (third column) and the mean value coordinates (fourth column) still lead to perfect predictions.
• Figure 5: We augment the training data by generating multiple cages with different offset values. This increases the robustness of CageNet against topological noise as the cage topology may change with different offsets, such as the hand region in this example.