Residual Primitive Fitting of 3D Shapes with SuperFrusta

TL;DR

This work tackles the reconstruction-editability gap by introducing SuperFrustum, an expressive, 8-parameter analytic primitive, and Residual Primitive Fitting (ResFit), an iterative procedure that couples global shape analysis with local optimization. By alternating between MSD-based initialization and gradient-based refinement, the method assembles compact, high-fidelity primitive programs that can be edited and reused as assets or CSG models. Across diverse benchmarks, it achieves state-of-the-art IoU while using roughly half the primitives, and enables downstream tasks such as image-to-assembly and semantic segmentation enrichment. Overall, the approach bridges dense 3D data and human-controllable design, enabling editable, interpretable shape programs with strong fidelity.

Abstract

We introduce a framework for converting 3D shapes into compact and editable assemblies of analytic primitives, directly addressing the persistent trade-off between reconstruction fidelity and parsimony. Our approach combines two key contributions: a novel primitive, termed SuperFrustum, and an iterative fiting algorithm, Residual Primitive Fitting (ResFit). SuperFrustum is an analytical primitive that is simultaneously (1) expressive, being able to model various common solids such as cylinders, spheres, cones & their tapered and bent forms, (2) editable, being compactly parameterized with 8 parameters, and (3) optimizable, with a sign distance field differentiable w.r.t. its parameters almost everywhere. ResFit is an unsupervised procedure that interleaves global shape analysis with local optimization, iteratively fitting primitives to the unexplained residual of a shape to discover a parsimonious yet accurate decompositions for each input shape. On diverse 3D benchmarks, our method achieves state-of-the-art results, improving IoU by over 9 points while using nearly half as many primitives as prior work. The resulting assemblies bridge the gap between dense 3D data and human-controllable design, producing high-fidelity and editable shape programs.

Figures (9)

• Figure 1: (Left) Primitive assemblies inferred by our method capture a wide range of shapes, including hollow forms (vase), curved & toroidal parts (bike), intricate geometry (ladder, robot), and smooth organic shapes (crab). (Right) Our approach shifts the reconstruction–parsimony Pareto frontier: compared to state-of-the-art methods, Marching Primitives marchingprim_Liu_2023CVPR (MPS) and Primitive Anything primitiveanything_ye_2025 (PA), we achieve markedly lower reconstruction error using significantly fewer primitives.
• Figure 2: SuperFrustum --- An Expressive, Compact & Differentiable Primitive. SuperFrustum is a unified analytic SDF primitive with only 8 parameters controlling dilation, taper, bulge, onion-like hollowing, profile roundness, and axial scaling. Its SDF is $C^0$-continuous and fully differentiable (almost eveywhere) with respect to all parameters, enabling robust inverse modeling and gradient-based optimization. As shown on the right, these parameters allow a single formulation to morph smoothly across common solids—cuboids, cylinders, cones, spheres, and toroidal variants—and to produce more complex shapes such as bent, hollow, or smoothly capped forms.
• Figure 3: ResFit infers parsimonious assemblies by interleaving shape analysis and primitive optimization. Shape decomposition provides initial primitives, which are refined with decomposition-aware optimization. Residual unexplained volumes are then extracted and seeded with new primitives.
• Figure 4: Morphological Shape Decomposition (MSD) iteratively extracts connected regions of similar thickness. Top: successive MSD partitions of a input mesh. Bottom: MSD yields regions that form suitable initialization seeds for SuperFrusta—capturing non-convex structures such as bicycle tires (left), a cat’s curved tail (center), and bowl rims (right). In contrast, CoACD over-partitions these regions into many convex fragments, often using axis-aligned cuts that produce semantically misaligned parts.
• Figure 5: Our method reconstructs target shapes with high geometric fidelity and produces more interpretable assemblies, using compact, minimally-overlapping primitives. In contrast, Primitive Anything primitiveanything_ye_2025 (PA) and Marching Primitives marchingprim_Liu_2023CVPR (MPS) often lose fine structure and generate assemblies with substantial primitive overlap.