RoboArmGS: High-Quality Robotic Arm Splatting via Bézier Curve Refinement

TL;DR

RoboArmGS addresses the challenge of creating high-fidelity digital twins of robotic arms by compensating for the gap between URDF-rigged motion and real-world dynamics. It introduces a hybrid representation that binds 3D Gaussians to a robot mesh via Structured Gaussian Binding and refines motion with a Bézier-based Motion Refiner, enabling coherent Gaussian binding and accurate motion across poses. The authors release RoboArm4D, a public dataset covering multiple arms with monocular videos, camera calibration, joint trajectories, and URDFs, providing a standardized benchmark. Experiments show state-of-the-art performance in novel-view and novel-pose rendering, with comprehensive ablations confirming the necessity of both SGB and BMR for high-fidelity dynamic assets. This work advances Real2Sim2Real pipelines by delivering controllable, photorealistic digital assets suitable for policy learning, control, and simulation.

Abstract

Building high-quality digital assets of robotic arms is crucial yet challenging for the Real2Sim2Real pipeline. Current approaches naively bind static 3D Gaussians according to URDF links, forcing them to follow an URDF-rigged motion passively. However, real-world arm motion is noisy, and the idealized URDF-rigged motion cannot accurately model it, leading to severe rendering artifacts in 3D Gaussians. To address these challenges, we propose RoboArmGS, a novel hybrid representation that refines the URDF-rigged motion with learnable Bézier curves, enabling more accurate real-world motion modeling. To be more specific, we present a learnable Bézier Curve motion refiner that corrects per-joint residuals to address mismatches between real-world motion and URDF-rigged motion. RoboArmGS enables the learning of more accurate real-world motion while achieving a coherent binding of 3D Gaussians across arm parts. To support future research, we contribute a carefully collected dataset named RoboArm4D, which comprises several widely used robotic arms for evaluating the quality of building high-quality digital assets. We evaluate our approach on RoboArm4D, and RoboArmGS achieves state-of-the-art performance in real-world motion modeling and rendering quality. The code and dataset will be released.

Figures (7)

• Figure 1: Qualitative results for Novel-View Synthesis. RoboArmGS synthesizes photorealistic images from unseen viewpoints for both the Universal Robots UR5e (left) and ABB IRB 120 (right). Our method faithfully reconstructs high-frequency details and preserves the visual fidelity of the robot's appearance, while maintaining sharp geometric boundaries against the background.
• Figure 2: Overview of RoboArmGS. Our method consists of two key modules: (a) Structured Gaussian Binding (SGB) binds 3D Gaussians according to the URDF links with adaptive densification; (b) Bézier-based Motion Refinement (BMR) corrects URDF-rigged motion and real-world motion discrepancies through learnable Bézier curves, enabling accurate novel-pose synthesis. The framework processes multi-view RGB images with known camera poses and robot joint angles, optimizing both modules jointly to achieve photorealistic rendering from novel viewpoints and robot motions.
• Figure 2: Qualitative results for Novel-Pose Synthesis. We visualize the rendered results of the robotic arms under unseen joint configurations (held-out test poses). Even in challenging poses that deviate significantly from the canonical training configurations, RoboArmGS maintains structural integrity and visual fidelity. The results demonstrate precise kinematic alignment and realistic shading changes consistent with the robot's motion.
• Figure 3: Qualitative comparison on novel-view synthesis. Our RoboArmGS achieves significantly sharper and more photorealistic renderings than Robo-GS lou2025robo, with better geometric accuracy and texture preservation.
• Figure 4: Qualitative comparison on novel-pose synthesis. RoboArmGS achieves photorealistic rendering with precise geometric alignment. Baselines fail in different ways: 4DGS and Deformable 3DGS produce blurred, distorted results lacking kinematic control; 3DGS+FK, despite correct Gaussian positioning, generates severe artifacts from inaccurate motion. Our BMR module successfully addresses both motion accuracy and rendering coherence.