PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with Auto-Regressive Transformer

TL;DR

PrimitiveAnything reframes 3D shape abstraction as a sequence generation task, learning from large-scale human-crafted primitive assemblies to produce human-aligned primitive decompositions across diverse categories. It combines an ambiguity-free primitive parameterization with a shape-conditioned decoder-only transformer that auto-regressively generates variable-length primitive sequences, guided by a point-cloud context. Trained on the HumanPrim dataset, the method achieves superior geometric fidelity and semantic alignment compared to optimization-based and learning-based baselines, and generalizes to ShapeNet and Objaverse. The approach enables efficient, editable, primitive-based 3D content generation with practical implications for games and user-generated content, while offering a modular framework extensible to new primitive types and contexts.

Abstract

Shape primitive abstraction, which decomposes complex 3D shapes into simple geometric elements, plays a crucial role in human visual cognition and has broad applications in computer vision and graphics. While recent advances in 3D content generation have shown remarkable progress, existing primitive abstraction methods either rely on geometric optimization with limited semantic understanding or learn from small-scale, category-specific datasets, struggling to generalize across diverse shape categories. We present PrimitiveAnything, a novel framework that reformulates shape primitive abstraction as a primitive assembly generation task. PrimitiveAnything includes a shape-conditioned primitive transformer for auto-regressive generation and an ambiguity-free parameterization scheme to represent multiple types of primitives in a unified manner. The proposed framework directly learns the process of primitive assembly from large-scale human-crafted abstractions, enabling it to capture how humans decompose complex shapes into primitive elements. Through extensive experiments, we demonstrate that PrimitiveAnything can generate high-quality primitive assemblies that better align with human perception while maintaining geometric fidelity across diverse shape categories. It benefits various 3D applications and shows potential for enabling primitive-based user-generated content (UGC) in games. Project page: https://primitiveanything.github.io

Figures (14)

• Figure 1: Overview. We propose PrimitiveAnything to decompose complex shapes into 3D primitive assembly via the auto-regressive transformer. Given human-crafted 3D primitive abstraction contents, we first design an ambiguity-free scheme to parameterize each primitive $p$ into class label $c$, translation $t$, rotation $r$ and scale $s$, and then employ a primitive encoder to form primitive token $h$. Meanwhile, a shape encoder encodes 3D shape features $\mathcal{C}$ from sampled point clouds. Our primitive transformer $\mathcal{S}$ predicts the next primitive based on the input condition $\mathcal{C}$ and previously generated primitives. To model the dependencies among primitive attributes, we proposed a cascaded primitive decoder $\mathcal{D}$ that sequentially predicts primitive attributes.
• Figure 2: Demonstration of primitive attribute ambiguity. A primitive with inherent symmetry can correspond to multiple scales and rotations through self-rotation and possible axis swapping.
• Figure 3: Qualitative comparisons on the HumanPrim test set: In our method, colors indicate different primitive types, while in Marching Primitives and EMS, colors represent separate superquadrics. Our method achieves human-crafted primitive abstraction and faithfully reproduces the original 3D structure.
• Figure 4: PrimitiveAnything interfaces with state-of-the-art 3D shape generation models to enable text- and image-conditioned primitive-based 3D content generation.
• Figure 5: More qualitative comparisons with optimization-based methods on the HumanPrim dataset.