Universal Online Temporal Calibration for Optimization-based Visual-Inertial Navigation Systems

TL;DR

The paper addresses the critical challenge of time misalignment between cameras and IMUs in visual-inertial navigation. It introduces a universal online temporal calibration by treating the time offset $t_d$ as a state, interpolating IMU poses to image timestamps using $t_d$ and estimated velocities to form a compensated visual residual, and enabling joint optimization of motion and time offset. The approach supports both 3D-position and depth parameterizations, derives Jacobians with respect to $t_d$, and integrates seamlessly into existing optimization-based VINS back-ends. Experimental results on EuRoC and simulations show robust online $t_d$ estimation and accurate ego-motion under noisy conditions, with broad applicability to VI-SLAM and LIO systems and comparable efficiency to traditional VINS pipelines.

Abstract

6-Degree of Freedom (6DoF) motion estimation with a combination of visual and inertial sensors is a growing area with numerous real-world applications. However, precise calibration of the time offset between these two sensor types is a prerequisite for accurate and robust tracking. To address this, we propose a universal online temporal calibration strategy for optimization-based visual-inertial navigation systems. Technically, we incorporate the time offset td as a state parameter in the optimization residual model to align the IMU state to the corresponding image timestamp using td, angular velocity and translational velocity. This allows the temporal misalignment td to be optimized alongside other tracking states during the process. As our method only modifies the structure of the residual model, it can be applied to various optimization-based frameworks with different tracking frontends. We evaluate our calibration method with both EuRoC and simulation data and extensive experiments demonstrate that our approach provides more accurate time offset estimation and faster convergence, particularly in the presence of noisy sensor data.

Figures (4)

• Figure 1: Timestamp misalignment in VINS. Using the $j^{th}$ image measurements to directly construct the residual model with the $j^{th}$ misaligned key state ($I_j$), ignoring the time offset $t_d$, will cause obvious inaccuracy because the $j^{th}$ image measurements are actually taken at the timestamp of the $j^{th}$ expected key state ($I^-_j$) considering the time offset $t_d$.
• Figure 2: Timestamp misalignment in VINS. In the sliding window, the reference time axis is defined with IMU time axis. When the $j^{th}$ image is coming, the key state in sliding window at ($t_{image} + t_{d_{j-1}}$) will be built and this key state will be mounted with this camera measurements to reduce the impact of timestamp misalignment as possible in raw data assignment.
• Figure 3: Time offset estimation in simulation with 20ms temporal offset. Estimated offset converges to more accurate value quickly within a few seconds.
• Figure 4: Time offset estimation in simulation with 40ms temporal offset. Estimated offset converges to more accurate value quickly within a few seconds.