iKalibr-RGBD: Partially-Specialized Target-Free Visual-Inertial Spatiotemporal Calibration For RGBDs via Continuous-Time Velocity Estimation

TL;DR

This letter presents an ego-velocity-estimation-based RGBD-inertial spatiotemporal calibrator, termed as iKalibr-RGBD, which is also targetless and continuous-time-based, but computationally efficient.

Abstract

Visual-inertial systems have been widely studied and applied in the last two decades (from the early 2000s to the present), mainly due to their low cost and power consumption, small footprint, and high availability. Such a trend simultaneously leads to a large amount of visual-inertial calibration methods being presented, as accurate spatiotemporal parameters between sensors are a prerequisite for visual-inertial fusion. In our previous work, i.e., iKalibr, a continuous-time-based visual-inertial calibration method was proposed as a part of one-shot multi-sensor resilient spatiotemporal calibration. While requiring no artificial target brings considerable convenience, computationally expensive pose estimation is demanded in initialization and batch optimization, limiting its availability. Fortunately, this could be vastly improved for the RGBDs with additional depth information, by employing mapping-free ego-velocity estimation instead of mapping-based pose estimation. In this paper, we present the continuous-time ego-velocity estimation-based RGBD-inertial spatiotemporal calibration, termed as iKalibr-RGBD, which is also targetless but computationally efficient. The general pipeline of iKalibr-RGBD is inherited from iKalibr, composed of a rigorous initialization procedure and several continuous-time batch optimizations. The implementation of iKalibr-RGBD is open-sourced at (https://github.com/Unsigned-Long/iKalibr) to benefit the research community.

Figures (8)

• Figure 1: Runtime visualization of visual-inertial spatiotemporal calibration in iKalibr, where two cameras and an IMU are involved. When depth information is available for all cameras (e.g., RGBDs), the velocity B-spline, instead of the translation B-spline, would be maintained in the estimator for continuous-time ego-velocity estimation.
• Figure 2: Illustration of first-order visual kinematics. Three-dimensional linear velocities of landmarks with respect to the camera, i.e., ${\boldsymbol{\mathrm{v}}_{l}^{c}}(\tau)$, are projected onto the image plane as corresponding two-dimensional pixel velocities $\dot{{\boldsymbol{\mathrm{f}}}}_l^c(\tau)$.
• Figure 3: Illustration of the pipeline of the proposed RGBD-Inertial spatiotemporal calibration.
• Figure 4: Schematic of two-dimensional pixel velocities computed using three-point first-order Lagrange polynomial. Red solid curve represents the feature trajectory from the (zero-order) Lagrange polynomial, while green line is the computed pixel velocity of the middle point (the feature indexed as $1$).
• Figure 5: The sensor suite and five scenarios in the real-world experiments. Datasets are from the open-source VECtor Benchmarkgao2022vector.