KineDiff3D: Kinematic-Aware Diffusion for Category-Level Articulated Object Shape Reconstruction and Generation

TL;DR

KineDiff3D tackles single-view category-level articulated object reconstruction by unifying geometry and kinematics within a diffusion framework. It introduces a Kinematic-Aware VAE (KA-VAE) to encode $SDF$ geometry, joint angles $A$, and segmentation into a latent $Z$, and deploys two conditional diffusion models: one for estimating the base pose in $SE(3)$ and joint parameters, and another for generating the kinematic-aware latent code from partial observations. An iterative Joint-Centric Chamfer Optimization refines pose, joints, and geometry while preserving articulation constraints, enabling both accurate reconstruction and generation of novel articulated states by manipulating $A$. Across synthetic, semi-synthetic, and real-world datasets, the approach demonstrates superior reconstruction fidelity, accurate kinematic estimation, and robust generalization, highlighting practical impact for robotics, AR, and vision systems dealing with articulated objects. Key contributions include the KA-VAE prior, joint-centric diffusion for pose and latent generation, and a diffusion-guided, Chamfer-based optimization loop that enforces articulation. The work advances real-time, single-view articulated-object understanding and controllable generation, with strong cross-domain transfer capabilities.

Abstract

Articulated objects, such as laptops and drawers, exhibit significant challenges for 3D reconstruction and pose estimation due to their multi-part geometries and variable joint configurations, which introduce structural diversity across different states. To address these challenges, we propose KineDiff3D: Kinematic-Aware Diffusion for Category-Level Articulated Object Shape Reconstruction and Generation, a unified framework for reconstructing diverse articulated instances and pose estimation from single view input. Specifically, we first encode complete geometry (SDFs), joint angles, and part segmentation into a structured latent space via a novel Kinematic-Aware VAE (KA-VAE). In addition, we employ two conditional diffusion models: one for regressing global pose (SE(3)) and joint parameters, and another for generating the kinematic-aware latent code from partial observations. Finally, we produce an iterative optimization module that bidirectionally refines reconstruction accuracy and kinematic parameters via Chamfer-distance minimization while preserving articulation constraints. Experimental results on synthetic, semi-synthetic, and real-world datasets demonstrate the effectiveness of our approach in accurately reconstructing articulated objects and estimating their kinematic properties.

Figures (6)

• Figure 1: KineDiff3D demonstrates strong generalization across datasets (synthetic, semi-synthetic, real-world), achieving robust reconstruction and novel articulated shape generation from single-view depth inputs.
• Figure 2: Pipeline of the KineDiff3D Framework. This framework integrates three synergistic modules: (a) Shape Prior Learning Module (top): Given a full object point cloud and its joint angles, a PointNet‑based KA‑VAE learns a shape prior embedding. (b) Pose and Joint Estimation Module (center): Employs a conditional diffusion model to jointly optimize base pose (X) and joint parameters (Y) during the denoising process. (c) Reconstruction Module (bottom): Learns the KA-VAE encoded shape prior embedding through diffusion modeling, then combines angle encoding with KA-VAE decoding via a kinematics-constrained articulation decoder to generate reconstructed articulated meshes.
• Figure 3: Kinematic Iterative Optimization Pipeline. Joint-Centric Chamfer Alignment iteratively refines SE(3) part poses, joint parameters and reconstruction precision.
• Figure 4: Qualitative Results on the Dataset ArtImage. Visual results demonstrate KineDiff3D's exceptional geometric fidelity in articulated object reconstruction and plausible generation, while baselines A-SDF and CARTO exhibit characteristic failures and lack segmentation capabilities.
• Figure 4: Quantitative Results on the Semi-Synthetic and Real-world Datasets.