ARC-Calib: Autonomous Markerless Camera-to-Robot Calibration via Exploratory Robot Motions

TL;DR

ARC-Calib tackles eye-to-hand calibration without markers or learning, addressing poor generalization in prior markerless methods by using self-generated visual patterns from exploratory robot motions. It combines a motion-planning module that yields trackable 2D ellipses and a geometry-based calibration module that enforces coplanarity and collinearity constraints to estimate $T_{bc}$ via a convex formulation and SVD-based projection. The approach is validated in both simulation and real-world FRANKA setups, showing robust convergence and superior IoU metrics (e.g., average IoU $\approx 0.94$ vs $0.84$) with relatively few motions (~26). These results demonstrate a practical, scalable calibration solution for vision-based manipulation on edge devices, eliminating the need for environmental markers or pre-trained models.

Abstract

Camera-to-robot (also known as eye-to-hand) calibration is a critical component of vision-based robot manipulation. Traditional marker-based methods often require human intervention for system setup. Furthermore, existing autonomous markerless calibration methods typically rely on pre-trained robot tracking models that impede their application on edge devices and require fine-tuning for novel robot embodiments. To address these limitations, this paper proposes a model-based markerless camera-to-robot calibration framework, ARC-Calib, that is fully autonomous and generalizable across diverse robots and scenarios without requiring extensive data collection or learning. First, exploratory robot motions are introduced to generate easily trackable trajectory-based visual patterns in the camera's image frames. Then, a geometric optimization framework is proposed to exploit the coplanarity and collinearity constraints from the observed motions to iteratively refine the estimated calibration result. Our approach eliminates the need for extra effort in either environmental marker setup or data collection and model training, rendering it highly adaptable across a wide range of real-world autonomous systems. Extensive experiments are conducted in both simulation and the real world to validate its robustness and generalizability.

Figures (9)

• Figure 1: Our camera-to-robot calibration framework estimates the pose transformation from the camera to the robot via exploratory robot motions. The method relies on analyzing the correspondence between the visual patterns extracted from keypoint trajectories and the robot motion.
• Figure 2: Green overlay indicates parts of the robot that are considered as the rotation of a single rigid body. The motion could be parameterized with the rotation axis $\mathcal{\vec{A}}^b_i$ and the joint position $\mathcal{D}^b_i$
• Figure 3: (left) Tracked visual features moving along 3D circles. (right) Samples of keypoint trajectories (red) and corresponding visual patterns (green) from conic functions fitting.
• Figure 4: A 3D cone (green) indicates the possible 3D circle candidates (blue and yellow). When projected onto the image plane, these candidates generate the visual pattern of a 2D ellipse (red)
• Figure 5: Perspective projection of the keypoint trajectories $\mathcal{P}^\pi_{i,j}$ on projection plane $\sigma_{i,l}$ create the projected trajectories $\mathcal{P}^{\sigma_{i,l}}_{i,j}$. The projected trajectories create a visual pattern of 2D circles that could be used to determine the reference positions of the exploratory motion.