Learning Convex Decomposition via Feature Fields

TL;DR

This work proposes a new formulation to the long-standing problem of convex decomposition through learning feature fields, enabling the first feed-forward model for open-world convex decomposition, and unlocks scalable, self-supervised learning on large datasets resulting in the first learned open-world model for convex decomposition.

Abstract

This work proposes a new formulation to the long-standing problem of convex decomposition through learning feature fields, enabling the first feed-forward model for open-world convex decomposition. Our method produces high-quality decompositions of 3D shapes into a union of convex bodies, which are essential to accelerate collision detection in physical simulation, amongst many other applications. The key insight is to adopt a feature learning approach and learn a continuous feature field that can later be clustered to yield a good convex decomposition via our self-supervised, purely-geometric objective derived from the classical definition of convexity. Our formulation can be used for single shape optimization, but more importantly, feature prediction unlocks scalable, self-supervised learning on large datasets resulting in the first learned open-world model for convex decomposition. Experiments show that our decompositions are higher-quality than alternatives and generalize across open-world objects as well as across representations to meshes, CAD models, and even Gaussian splats. https://research.nvidia.com/labs/sil/projects/learning-convex-decomp/

Figures (14)

• Figure 1: Our method takes an input shape (top), infers features from an open-world model learned with our new self-supervised geometric loss (middle), and clusters those features to fit the shape with a collection of tight convex bounding proxies (bottom).
• Figure 2: By adjusting the clustering threshold, our method can generate decompositions at varying granularity, all from the same feature field.
• Figure 3: An overview of our convex decomposition pipeline. We train a feedforward model that takes a point-sampled 3D shape as input and predicts a feature field represented defined over the object. At training time, these features are fit with a self-supervised geometric objective derived from the definition of convexity. At inference time, the features are clustered to split the shape into components, and the convex hull of each component becomes the decomposition. Note that feature colors are visualized by running PCA on the feature field.
• Figure 4: Contrastive training triplets are formed by a source point, a positive pair generated by casting a ray in a random inward direction (left), and a negative pair rejection-sampled from all points on the surface weighted to prefer nearby points (right).
• Figure 5: Left: Visualization of the learned feature field during our single-shape optimization, shown at different optimization stages. Right: Feature fields produced directly by our feed-forward model.