SurgCalib: Gaussian Splatting-Based Hand-Eye Calibration for Robot-Assisted Minimally Invasive Surgery

TL;DR

SurgCalib is proposed, an automatic, markerless framework that has the potential to be used in the OR and first initializes the pose of the surgical instrument using raw kinematic measurements and subsequently refines this pose through a two-phase optimization procedure under the RCM constraint within a Gaussian Splatting-based differentiable rendering pipeline.

Abstract

We present a Gaussian Splatting-based framework for hand-eye calibration of the da Vinci surgical robot. In a vision-guided robotic system, accurate estimation of the rigid transformation between the robot base and the camera frame is essential for reliable closed-loop control. For cable-driven surgical robots, this task faces unique challenges. The encoders of surgical instruments often produce inaccurate proprioceptive measurements due to cable stretch and backlash. Conventional hand-eye calibration approaches typically rely on known fiducial patterns and solve the AX = XB formulation. While effective, introducing additional markers into the operating room (OR) environment can violate sterility protocols and disrupt surgical workflows. In this study, we propose SurgCalib, an automatic, markerless framework that has the potential to be used in the OR. SurgCalib first initializes the pose of the surgical instrument using raw kinematic measurements and subsequently refines this pose through a two-phase optimization procedure under the RCM constraint within a Gaussian Splatting-based differentiable rendering pipeline. We evaluate the proposed method on the public dVRK benchmark, SurgPose. The results demonstrate average 2D tool-tip reprojection errors of 12.24 px (2.06 mm) and 11.33 px (1.9 mm), and 3D tool-tip Euclidean distance errors of 5.98 mm and 4.75 mm, for the left and right instruments, respectively.

Figures (7)

• Figure 1: (a) The diagram of task definition. The RCM point lies within the yellow dashed circle. The shaft centerlines are shown as green lines. (b) The frame definition of the EndoWrist surgical instrument.
• Figure 2: The schematic of our proposed pose initialization and refinement method.
• Figure 3: Example images rendered by Instrument-Splatting.
• Figure 4: The trajectories of end-effector origins. From left to right: raw data from the dVRK, estimated positions by the proposed pose refinement approach, and the aligned trajectories using hand-eye transformation.
• Figure 5: The visualization of the compensated end-effector pose. The green and purple dots are the ground truth and reprojected tool tips, respectively. The upper and bottom row refers to frames with fewer and larger errors.