RoMeO: Robust Metric Visual Odometry

TL;DR

RoMeO tackles monocular visual odometry without IMUs by leveraging priors from pretrained depth models to recover metric-scale trajectories and improve robustness. It introduces Robust Depth-Guided Bundle Adjustment, integrating both monocular metric depth and MVS depth priors into the BA objective, with adaptive depth regularization and a flow-based refinement loop. Noise Augmented Training aligns the flow network to depth-enhanced inputs while preserving robustness to depth noise, and MVS guidance is selectively activated based on motion criteria and masked by confidence maps. Across six diverse zero-shot datasets, RoMeO achieves large improvements over SOTA methods in both VO and SLAM, demonstrating strong generalization and practical impact for outdoor and indoor scenarios.

Abstract

Visual odometry (VO) aims to estimate camera poses from visual inputs -- a fundamental building block for many applications such as VR/AR and robotics. This work focuses on monocular RGB VO where the input is a monocular RGB video without IMU or 3D sensors. Existing approaches lack robustness under this challenging scenario and fail to generalize to unseen data (especially outdoors); they also cannot recover metric-scale poses. We propose Robust Metric Visual Odometry (RoMeO), a novel method that resolves these issues leveraging priors from pre-trained depth models. RoMeO incorporates both monocular metric depth and multi-view stereo (MVS) models to recover metric-scale, simplify correspondence search, provide better initialization and regularize optimization. Effective strategies are proposed to inject noise during training and adaptively filter noisy depth priors, which ensure the robustness of RoMeO on in-the-wild data. As shown in Fig.1, RoMeO advances the state-of-the-art (SOTA) by a large margin across 6 diverse datasets covering both indoor and outdoor scenes. Compared to the current SOTA DPVO, RoMeO reduces the relative (align the trajectory scale with GT) and absolute trajectory errors both by >50%. The performance gain also transfers to the full SLAM pipeline (with global BA & loop closure). Code will be released upon acceptance.

Figures (5)

• Figure 1: Teaser.(a) RoMeO vs SOTA methods across 6 diverse datasets covering 3 indoor and 3 outdoor scenes, RoMeO outperforms SOTA methods by a large margin both in terms of the trajectory shape (relative trajectory error (RTE)) and scale (absolute trajectory error (ATE)). (b) RoMeO trajectories closely align with the ground truth, even without scale alignment.
• Figure 2: Overview. RoMeO initializes each frame using monocular metric depth models. MVS models are used to further refine intermediate BA depth. Besides replacing the initial/intermediate depth, monocular and MVS depth priors are also added into the regularization terms of BA, with adaptive conditions to filter noisy depth priors and enable effective MVS prediction. Noise augmented training is used to adapt the flow network to depth-enhanced inputs, which maximizes the accuracy while maintaining the robustness to prior noise.
• Figure 3: Residual flow magnitudes in different BA iterations.
• Figure 4: Trajectory. For each scene, the two rows show respectively the trajectory with and without scale alignment. RoMeO aligns much better with GT both with and without scale alignment.
• Figure 5: Point cloud visualization. First two columns: results on KITTI Odometry. Last three columns: results on TUM-RGBD. RoMeO provides dense and more accurate 3D reconstructions.