Less Biased Noise Scale Estimation for Threshold-Robust RANSAC

TL;DR

This work tackles the challenge of selecting a robust inlier threshold for RANSAC in relative pose estimation. It revisits SIMFIT, identifies biases in noise-scale estimation, and introduces SIMFIT++ with a train/validation split and a threshold-aware debiasing scheme, plus a multi-pair extension to integrate information across image pairs. The method maps the inlier noise scale to the RANSAC threshold via $\tau = F^{-1}(\alpha)\sigma$ with $\alpha=0.99$, and demonstrates improved performance over fixed thresholds and the original SIMFIT across essential and fundamental matrix estimation on PoseBench datasets. The proposed approach increases threshold robustness in practice, reducing sensitivity to dataset-specific noise and enabling more reliable relative pose estimation in diverse scenes, though it acknowledges potential biases and runtime considerations as areas for future work.

Abstract

The gold-standard for robustly estimating relative pose through image matching is RANSAC. While RANSAC is powerful, it requires setting the inlier threshold that determines whether the error of a correspondence under an estimated model is sufficiently small to be included in its consensus set. Setting this threshold is typically done by hand, and is difficult to tune without an access to ground truth data. Thus, a method capable of automatically determining the optimal threshold would be desirable. In this paper we revisit inlier noise scale estimation, which is an attractive approach as the inlier noise scale is linear to the optimal threshold. We revisit the noise scale estimation method SIMFIT and find bias in the estimate of the noise scale. In particular, we fix underestimates from using the same data for fitting the model as estimating the inlier noise, and from not taking the threshold itself into account. Secondly, since the optimal threshold within a scene is approximately constant we propose a multi-pair extension of SIMFIT++, by filtering of estimates, which improves results. Our approach yields robust performance across a range of thresholds, shown in Figure 1. Code is available at https://github.com/Parskatt/simfitpp

Figures (10)

• Figure 1: Fundamental matrix estimation performance as a function of RANSAC threshold (higher is better). Traditionally, following an initial guess, RANSAC thresholds are not updated during optimization. In practice, this means that the threshold will be suboptimal, as different images and models require different thresholds. We overcome this issue by alternating between estimating the noise scale $\sigma$ and RANSAC, and using that $\tau = F^{-1}(\alpha) \sigma$ for the confidence level $\alpha \in [0,1]$, which we fix as $\alpha=0.99$.
• Figure 2: Qualitative comparison of fit to empirical residual distribution for SuperGlue. We show the distribution of squared residual errors for SuperPoint detone2018superpoint with SuperGlue sarlin2020superglue on a randomly selected pair in the MegaDepth1500 sun2021loftr test set.
• Figure 3: Qualitative comparison of fit to empirical residual distribution for SuperGlue. We show the distribution of squared residual errors for SuperPoint detone2018superpoint with SuperGlue sarlin2020superglue on a randomly selected pair in the ScanNet dai2017scannet test set.
• Figure 4: Qualitative comparison of fit to empirical residual distribution for RoMa. We show the distribution of squared residual errors for RoMa edstedt2024roma on a randomly selected pair in the MegaDepth1500 sun2021loftr test set.
• Figure 5: Qualitative comparison of fit to empirical residual distribution for RoMa. We show the distribution of squared residual errors for RoMa edstedt2024roma on a randomly selected pair in the ScanNet dai2017scannet test set.

Theorems & Definitions (1)

• Definition 2.1