RigMo: Unifying Rig and Motion Learning for Generative Animation

TL;DR

RigMo addresses the gap of independently modeling rig structure and motion in 4D animation by proposing a unified, self-supervised framework that learns from raw deforming meshes. It factorizes deformation into two latent spaces: a rig latent that decodes to Gaussian bones and skinning weights and a motion latent that yields time-varying SE(3) transforms, enabling fully animatable meshes without ground-truth rigs. The RigMo-VAE, with a topology-aware encoder and Gaussian Skinning LBS, is complemented by Motion-DiT, a diffusion transformer operating in the motion latent space for controllable generation. Across diverse datasets, RigMo demonstrates interpretable rigs, smooth deformations, and strong generalization, offering a scalable alternative to manual rigging and per-sequence optimization for deformable 3D animation.

Abstract

Despite significant progress in 4D generation, rig and motion, the core structural and dynamic components of animation are typically modeled as separate problems. Existing pipelines rely on ground-truth skeletons and skinning weights for motion generation and treat auto-rigging as an independent process, undermining scalability and interpretability. We present RigMo, a unified generative framework that jointly learns rig and motion directly from raw mesh sequences, without any human-provided rig annotations. RigMo encodes per-vertex deformations into two compact latent spaces: a rig latent that decodes into explicit Gaussian bones and skinning weights, and a motion latent that produces time-varying SE(3) transformations. Together, these outputs define an animatable mesh with explicit structure and coherent motion, enabling feed-forward rig and motion inference for deformable objects. Beyond unified rig-motion discovery, we introduce a Motion-DiT model operating in RigMo's latent space and demonstrate that these structure-aware latents can naturally support downstream motion generation tasks. Experiments on DeformingThings4D, Objaverse-XL, and TrueBones demonstrate that RigMo learns smooth, interpretable, and physically plausible rigs, while achieving superior reconstruction and category-level generalization compared to existing auto-rigging and deformation baselines. RigMo establishes a new paradigm for unified, structure-aware, and scalable dynamic 3D modeling.

Figures (6)

• Figure 1: RigMo jointly learns rigging and motion by understanding the underlying structure of mesh sequences. Unlike optimization-based methods that fit a rig per sequence, RigMo is a feed-forward framework that infers Gaussian bones, skinning weights, and motion parameters directly from input meshes for unified 4D animation generation. Colors visualize the influence of Gaussian bones on vertices (skinning weights) across the mesh surface; similar colors may appear for different bones as they are randomly assigned for visualization.
• Figure 2: Overview of the RigMo-VAE framework. Given temporal vertex trajectories from deforming mesh sequences, RigMo employs a dual-path encoder to disentangle static geometry (rigging branch) and dynamic motion (motion branch), learning a compact latent representation that captures both spatial structure and temporal dynamics. The decoder maps these latent features to physically interpretable rig components: Gaussian bone descriptors defining geodesic-aware skinning weights and variational motion parameters for local and root transformations. Different colors indicate the influence regions of learned Gaussian bones, demonstrating semantically meaningful decomposition of mesh deformation without manual rigging supervision.
• Figure 3: Overview of the Motion DiT. Given static rigging features, a condition encoder produces anchor and global tokens that guide a diffusion transformer operating in RigMo’s motion-latent space. The model uses spatial, temporal, and frame-conditioned cross-attention to predict denoised motion latents, which are decoded into bone transformations and vertex sequences via Gaussian skinning.
• Figure 4: Results produced by the full RigMo. Given a sparse input sequence, where a subset of frames is observed according to a frame mask, RigMo reconstructs a complete animatable model by jointly predicting the rigging structure (Gaussian bones and skinning weights) and synthesizing the missing motion frames through diffusion in the RigMo latent space. The resulting rigged model produces coherent, articulated motion across humans, animals, and diverse non-human shapes, demonstrating that sparse observations are sufficient to recover a full animation without category-specific priors.
• Figure 5: Comparison between UniRig+Optimization and our RigMo Rigging Module. Although UniRig may produce visually plausible skinning weights in some cases (e.g., the fox), its rigging does not generalize and collapses under actual animation, leading to severe deformation artifacts. In contrast, RigMo learns robust and transferable rig structures directly from motion, without any ground-truth rig supervision, and achieves stable, high-fidelity deformations across diverse poses and animal species.