REArtGS: Reconstructing and Generating Articulated Objects via 3D Gaussian Splatting with Geometric and Motion Constraints

TL;DR

The paper tackles the challenge of reconstructing and generating textured surfaces for articulated objects using only RGB views from two states. It introduces REArtGS, which first leverages an unbiased SDF-guided regularization to link Gaussian opacity to the true surface, and then learns motion-constrained deformable fields aligned with articulated kinematics to synthesize unseen states in an unsupervised manner. The approach yields high-quality texture-rich meshes and accurate unseen-state generation, outperforming state-of-the-art methods on PartNet-Mobility and AKB-48 across reconstruction and generation tasks. This has practical implications for robotics and AR/VR, enabling robust 3D understanding and animation of articulated objects from limited multi-view data. Future work targets pose estimation without explicit camera priors and handling transparent materials to broaden applicability.

Abstract

Articulated objects, as prevalent entities in human life, their 3D representations play crucial roles across various applications. However, achieving both high-fidelity textured surface reconstruction and dynamic generation for articulated objects remains challenging for existing methods. In this paper, we present REArtGS, a novel framework that introduces additional geometric and motion constraints to 3D Gaussian primitives, enabling realistic surface reconstruction and generation for articulated objects. Specifically, given multi-view RGB images of arbitrary two states of articulated objects, we first introduce an unbiased Signed Distance Field (SDF) guidance to regularize Gaussian opacity fields, enhancing geometry constraints and improving surface reconstruction quality. Then we establish deformable fields for 3D Gaussians constrained by the kinematic structures of articulated objects, achieving unsupervised generation of surface meshes in unseen states. Extensive experiments on both synthetic and real datasets demonstrate our approach achieves high-quality textured surface reconstruction for given states, and enables high-fidelity surface generation for unseen states. Project site: https://sites.google.com/view/reartgs/home.

Figures (6)

• Figure 1: Given multi-view RGB images of articulated objects from two arbitrary states, our REArtGS enables high-quality textured surface mesh reconstruction and generation for unseen states.
• Figure 2: The overall pipeline of REArtGS. We introduce additional geometric and motion constraints for 3D Gaussian primitives, achieving high-quality surface mesh reconstruction and time-continuous generation, with only multi-view images from arbitrary two states.
• Figure 3: The illustration of unbiased SDF regularization. It can be observed that when $t$ approaches $t^{*}$, the absolute value of SDF $|\mathcal{S}|$ converges to zero with the unbiased SDF regularization.
• Figure 4: The qualitative result of surface reconstruction on PartNet-Mobility dataset. We show both textured and non-textured meshes for the best comparison.
• Figure 5: The qualitative results of surface generation at arbitrary unseen states on PartNet-Mobility dataset. We show both textured and non-textured meshes for best comparison. The states are sampled randomly.