ARMO: Autoregressive Rigging for Multi-Category Objects

TL;DR

ARMO addresses the need for scalable rigging of diverse 3D shapes by introducing OmniRig, a large-scale rigging dataset, and a novel autoregressive rigging framework that jointly predicts joint positions and bone connectivity. The pipeline tokenizes skeletons as a complete graph into $6k$ tokens, encodes them with an autoregressive auto-encoder, and uses a mesh-conditioned latent diffusion model to align skeletons to meshes, reducing error propagation common in regression-based methods. On OmniRig, ARMO achieves state-of-the-art performance in skeleton prediction across diverse categories and demonstrates a practical motion-transfer application, enabling automatic animation for arbitrary shapes. This combination of dataset and end-to-end rigging methodology provides a robust foundation for dynamic 3D content generation and animation synthesis.

Abstract

Recent advancements in large-scale generative models have significantly improved the quality and diversity of 3D shape generation. However, most existing methods focus primarily on generating static 3D models, overlooking the potentially dynamic nature of certain shapes, such as humanoids, animals, and insects. To address this gap, we focus on rigging, a fundamental task in animation that establishes skeletal structures and skinning for 3D models. In this paper, we introduce OmniRig, the first large-scale rigging dataset, comprising 79,499 meshes with detailed skeleton and skinning information. Unlike traditional benchmarks that rely on predefined standard poses (e.g., A-pose, T-pose), our dataset embraces diverse shape categories, styles, and poses. Leveraging this rich dataset, we propose ARMO, a novel rigging framework that utilizes an autoregressive model to predict both joint positions and connectivity relationships in a unified manner. By treating the skeletal structure as a complete graph and discretizing it into tokens, we encode the joints using an auto-encoder to obtain a latent embedding and an autoregressive model to predict the tokens. A mesh-conditioned latent diffusion model is used to predict the latent embedding for conditional skeleton generation. Our method addresses the limitations of regression-based approaches, which often suffer from error accumulation and suboptimal connectivity estimation. Through extensive experiments on the OmniRig dataset, our approach achieves state-of-the-art performance in skeleton prediction, demonstrating improved generalization across diverse object categories. The code and dataset will be made public for academic use upon acceptance.

Figures (6)

• Figure 1: Visualization of the dataset $\texttt{OmniRig}$, which contains high-quality skeleton structures and objects in diverse categories.
• Figure 2: A pie chart indicating the multiple categories in our largr-scale rigging dataset $\texttt{OmniRig}$.
• Figure 3: The overall pipeline of our framework. (a) An autoregressive auto-encoder model to establish the latent embedding for the skeleton. (b) A conditioned latent diffusion model to align the skeleton with the mesh through latent features. See Sec.\ref{['sec:skeleton']} for more details.
• Figure 4: Comparison of skeleton generation results on $\texttt{OmniRig}$. Our method can generate reasonable skeleton results for diverse object categories and inputs with complex poses.
• Figure 5: We present additional qualitative results of skeleton generation. Our model is capable of producing reasonable skeletal structures for inputs with diverse categories and varying poses.