SURE: Semi-dense Uncertainty-REfined Feature Matching

TL;DR

SURE is a Semi- dense Uncertainty-REfined matching framework that jointly predicts correspondences and their confidence by modeling both aleatoric and epistemic uncertainties and consistently outperforms existing state-of-the-art semi-dense matching models in both accuracy and efficiency.

Abstract

Establishing reliable image correspondences is essential for many robotic vision problems. However, existing methods often struggle in challenging scenarios with large viewpoint changes or textureless regions, where incorrect cor- respondences may still receive high similarity scores. This is mainly because conventional models rely solely on fea- ture similarity, lacking an explicit mechanism to estimate the reliability of predicted matches, leading to overconfident errors. To address this issue, we propose SURE, a Semi- dense Uncertainty-REfined matching framework that jointly predicts correspondences and their confidence by modeling both aleatoric and epistemic uncertainties. Our approach in- troduces a novel evidential head for trustworthy coordinate regression, along with a lightweight spatial fusion module that enhances local feature precision with minimal overhead. We evaluated our method on multiple standard benchmarks, where it consistently outperforms existing state-of-the-art semi-dense matching models in both accuracy and efficiency. our code will be available on https://github.com/LSC-ALAN/SURE.

Figures (5)

• Figure 1: Comparison of E-LoFTR and our method on the MegaDepth dataset. Lines highlighted in green and red correspond to points with an epipolar error less than or greater than $10^{-4}$ respectively.
• Figure 2: Overview of the proposed SURE framework. (1) A backbone extracts coarse features $F_c$ and a Spatial Fusion Module provides fine features $F_f$. (2) The coarse matching module produces initial correspondences $M_c$, which are used to sample $F_f$ for fine-level refinement. (3) Trustworthy Regression produces precise offsets $(\Delta x, \Delta y)$ along with uncertainty estimates. Specifically, an Evidential Head predicts the parameters $(\psi, \eta, \kappa, \rho)$ of a Normal-Inverse-Gamma distribution, which jointly encode the offset and its associated aleatoric and epistemic uncertainties.
• Figure 3: Qualitative comparisons of SURE against Light Glue and E-LoFTR on both indoor and outdoor scenes. SURE achieves a higher number of correct matches and reduces mismatches, demonstrating robustness in low-texture areas as well as under significant viewpoint and lighting variations. Red regions denote points with epipolar error exceeding $5 \times 10^{-4}$ for indoor and $1 \times 10^{-4}$ for outdoor scenes.
• Figure 4: We selected large viewpoint changes and weak-texture scenarios. Among 2048 correspondences, the 50 pairs with the highest model uncertainty and data uncertainty were chosen. The lighter the line color, the higher the uncertainty.
• Figure 5: Uncertainty analysis. (a) and (b) are the heat maps of the Spearman rank correlation analysis between the EPE and the uncertainties.