Articulation in Motion: Prior-free Part Mobility Analysis for Articulated Objects By Dynamic-Static Disentanglement

TL;DR

A new framework for articulation in Motion is presented, which infer part-level decomposition, articulation kinematics, and reconstruct an interactive 3D digital replica from a user-object interaction video and a start-state scan, and proposes a dual-Gaussian scene representation that uses motion cues to segment the object into parts and assign articulation joints.

Abstract

Articulated objects are ubiquitous in daily life. Our goal is to achieve a high-quality reconstruction, segmentation of independent moving parts, and analysis of articulation. Recent methods analyse two different articulation states and perform per-point part segmentation, optimising per-part articulation using cross-state correspondences, given a priori knowledge of the number of parts. Such assumptions greatly limit their applications and performance. Their robustness is reduced when objects cannot be clearly visible in both states. To address these issues, in this paper, we present a new framework, Articulation in Motion (AiM). We infer part-level decomposition, articulation kinematics, and reconstruct an interactive 3D digital replica from a user-object interaction video and a start-state scan. We propose a dual-Gaussian scene representation that is learned from an initial 3DGS scan of the object and a video that shows the movement of separate parts. It uses motion cues to segment the object into parts and assign articulation joints. Subsequently, a robust, sequential RANSAC is employed to achieve part mobility analysis without any part-level structural priors, which clusters moving primitives into rigid parts and estimates kinematics while automatically determining the number of parts. The proposed approach separates the object into parts, each represented as a 3D Gaussian set, enabling high-quality rendering. Our approach yields higher quality part segmentation than previous methods, without prior knowledge. Extensive experimental analysis on both simple and complex objects validates the effectiveness and strong generalisation ability of our approach. Project page: https://haoai-1997.github.io/AiM/.

Figures (20)

• Figure 1: Left: Prior two-state methods often degrade on the sequences from closed-start to open-end. Right: Results of the proposed AiM, compared to ground truth (GT) geometry.
• Figure 2: Left: DTA and ArtGS fail to recover from an incorrect input number of parts (4 here) and result in over-segmentations; Right: Visual results of DTA and ArtGS with closed-start and open-end states. The static part is gray and the moving part is green. In contrast, Ours requires no geometric priors and recovers accurate part-level segmentation from the continuous closed-start$\rightarrow$open-end interaction process.
• Figure 3: Overview of the first two stages: I) 3DGS start-state $\{\mathcal{G}^{S}\}$ reconstruction from a multi-view RGB scan. II) A deformable 3DGS $\{\mathcal{G}^{M},t\}$ tracks motion video, while joint optimisation prunes moving components from $\{\mathcal{G}^{S}\}$. Pruned static Gaussian set $\{\mathcal{G}^{S}_{p}\}$ encodes the static base. An SDMD module handles newly revealed but static Gaussians. Together, these yield two separated Gaussian sets ($\{\mathcal{G}^{S}_{p}\}$ and $\{\mathcal{G}^{M},t\}$) for the articulation analysis (Fig. \ref{['fig:framework_s3']}).
• Figure 4: Renderings of the start (left), and end (middle) states. Without SDMD detection, some newly revealed static parts are wrongly associated with the moving Gaussian set (right).
• Figure 5: Stage III: Motion-based part segmentation and articulation analysis. As the clean $\{\mathcal{G}^{M}, t\}$ provides time-varying trajectories, Sequential RANSAC groups trajectories into rigid parts (multi-part supported) without priors or optimisation, and directly outputs per-part articulation parameters. The green (top) and purple (bottom) points are our predicted moving Gaussians.