PD$^{2}$GS: Part-Level Decoupling and Continuous Deformation of Articulated Objects via Gaussian Splatting

TL;DR

PD$^{2}$GS, a novel framework that learns a shared canonical Gaussian field and models the arbitrary interaction state as its continuous deformation, jointly encoding geometry and kinematics, enables accurate and reliable part-level decoupling while enforcing mutual exclusivity between parts and preserving scene-level coherence.

Abstract

Articulated objects are ubiquitous and important in robotics, AR/VR, and digital twins. Most self-supervised methods for articulated object modeling reconstruct discrete interaction states and relate them via cross-state geometric consistency, yielding representational fragmentation and drift that hinder smooth control of articulated configurations. We introduce PD$^{2}$GS, a novel framework that learns a shared canonical Gaussian field and models the arbitrary interaction state as its continuous deformation, jointly encoding geometry and kinematics. By associating each interaction state with a latent code and refining part boundaries using generic vision priors, PD$^{2}$GS enables accurate and reliable part-level decoupling while enforcing mutual exclusivity between parts and preserving scene-level coherence. This unified formulation supports part-aware reconstruction, fine-grained continuous control, and accurate kinematic modeling, all without manual supervision. To assess realism and generalization, we release RS-Art, a real-to-sim RGB-D dataset aligned with reverse-engineered 3D models, supporting real-world evaluation. Extensive experiments demonstrate that PD$^{2}$GS surpasses prior methods in geometric and kinematic accuracy, and in consistency under continuous control, both on synthetic and real data.

Figures (16)

• Figure 1: Given multi-view image sets at arbitrary interaction states, our framework (a) builds a unified 3D Gaussian representation for articulated objects, (b) achieves part-level articulated object reconstruction and joint motion analysis, and (c) enables continuous deformation of articulated objects with multi-part decoupling by interpolating latent codes.
• Figure 2: Overview of the PD$^{2}$GS pipeline. Solid arrows indicate differentiable modules that participate in the joint optimization, whereas dashed arrows correspond to non-differentiable post-processing stages executed outside the optimization loop.
• Figure 3: Qualitative multi-task modeling results on multi-part articulated objects.
• Figure 4: Generalization to unseen interaction states. We interpolate the latent code while deforming only the Gaussians that belong to a chosen part.
• Figure 5: Qualitative reconstruction results on intermediate states.