Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AffineGlue: Joint Matching and Robust Estimation

Daniel Barath, Dmytro Mishkin, Luca Cavalli, Paul-Edouard Sarlin, Petr Hruby, Marc Pollefeys

TL;DR

AffineGlue addresses the challenge of jointly matching features and estimating two-view geometry by replacing exhaustive one-to-one matching with one-to-many correspondences and single-point minimal solvers. A key innovation is the 1AC-based minimal solvers, including a new homography solver that exploits a known gravity direction, and a neural network (NeFSAC) that reweights ACs by their likelihood of yielding a good model. The method couples NextBestMatch, guided matching, and local optimization to keep the search space tractable while maintaining accuracy, achieving state-of-the-art performance on PhotoTourism and competitive results on ScanNet, HPatches, and LoFTR-like baselines. The approach significantly improves robustness to matching ambiguities and demonstrates practical runtimes, with potential for integration with affine-aware feature pipelines in the future.

Abstract

We propose AffineGlue, a method for joint two-view feature matching and robust estimation that reduces the combinatorial complexity of the problem by employing single-point minimal solvers. AffineGlue selects potential matches from one-to-many correspondences to estimate minimal models. Guided matching is then used to find matches consistent with the model, suffering less from the ambiguities of one-to-one matches. Moreover, we derive a new minimal solver for homography estimation, requiring only a single affine correspondence (AC) and a gravity prior. Furthermore, we train a neural network to reject ACs that are unlikely to lead to a good model. AffineGlue is superior to the SOTA on real-world datasets, even when assuming that the gravity direction points downwards. On PhotoTourism, the AUC@10° score is improved by 6.6 points compared to the SOTA. On ScanNet, AffineGlue makes SuperPoint and SuperGlue achieve similar accuracy as the detector-free LoFTR.

AffineGlue: Joint Matching and Robust Estimation

TL;DR

AffineGlue addresses the challenge of jointly matching features and estimating two-view geometry by replacing exhaustive one-to-one matching with one-to-many correspondences and single-point minimal solvers. A key innovation is the 1AC-based minimal solvers, including a new homography solver that exploits a known gravity direction, and a neural network (NeFSAC) that reweights ACs by their likelihood of yielding a good model. The method couples NextBestMatch, guided matching, and local optimization to keep the search space tractable while maintaining accuracy, achieving state-of-the-art performance on PhotoTourism and competitive results on ScanNet, HPatches, and LoFTR-like baselines. The approach significantly improves robustness to matching ambiguities and demonstrates practical runtimes, with potential for integration with affine-aware feature pipelines in the future.

Abstract

We propose AffineGlue, a method for joint two-view feature matching and robust estimation that reduces the combinatorial complexity of the problem by employing single-point minimal solvers. AffineGlue selects potential matches from one-to-many correspondences to estimate minimal models. Guided matching is then used to find matches consistent with the model, suffering less from the ambiguities of one-to-one matches. Moreover, we derive a new minimal solver for homography estimation, requiring only a single affine correspondence (AC) and a gravity prior. Furthermore, we train a neural network to reject ACs that are unlikely to lead to a good model. AffineGlue is superior to the SOTA on real-world datasets, even when assuming that the gravity direction points downwards. On PhotoTourism, the AUC@10° score is improved by 6.6 points compared to the SOTA. On ScanNet, AffineGlue makes SuperPoint and SuperGlue achieve similar accuracy as the detector-free LoFTR.
Paper Structure (7 sections, 7 equations, 3 figures, 5 tables)

This paper contains 7 sections, 7 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Theoretical Background
  3. Joint Matching and Estimation
  4. Homography from 1AC
  5. Experiments
  6. Real-World Experiments
  7. Conclusion

Figures (3)

  • Figure 1: The steps of the AffineGlue pipeline are as follows: (1) features with affine shapes are detected in the input images, e.g., by SuperPoint SuperPoint2017 combined with AffNet AffNet2018. (2) For each feature in the source image, the matching by, e.g. SuperGlue sarlin2020superglue, is often ambiguous, especially, at repeated patterns. Thus, we form one-to-many matches for each point in the source image. (3) AffineGlue iteratively selects a candidate one-to-one affine correspondence and estimates the model (e.g., relative pose) by a single-point solver. Guided sampling then forms one-to-one correspondences consistent with the estimated model to calculate its score and select its inliers.
  • Figure 2: Stability study. The frequencies ($100$k runs) of $\text{log}_{10}$ rotation and translation errors (both in degrees) in the homography estimated by the 4PC hartley2003multiple, 2AC barath2017theory, and proposed 1AC+G(H) solvers.
  • Figure 3: Image noise study. The average (over $100$k runs) angular errors of the rotations and translation estimated by the 4PC hartley2003multiple, 2AC barath2017theory, and proposed 1AC+G(H) homography solvers plotted as a function of the image noise in pixels.