HCPM: Hierarchical Candidates Pruning for Efficient Detector-Free Matching

TL;DR

HCPM is an efficient and detector-free local feature-matching method that employs hierarchical pruning to optimize the matching pipeline and significantly surpasses existing methods in terms of speed while maintaining high accuracy.

Abstract

Deep learning-based image matching methods play a crucial role in computer vision, yet they often suffer from substantial computational demands. To tackle this challenge, we present HCPM, an efficient and detector-free local feature-matching method that employs hierarchical pruning to optimize the matching pipeline. In contrast to recent detector-free methods that depend on an exhaustive set of coarse-level candidates for matching, HCPM selectively concentrates on a concise subset of informative candidates, resulting in fewer computational candidates and enhanced matching efficiency. The method comprises a self-pruning stage for selecting reliable candidates and an interactive-pruning stage that identifies correlated patches at the coarse level. Our results reveal that HCPM significantly surpasses existing methods in terms of speed while maintaining high accuracy. The source code will be made available upon publication.

Figures (5)

• Figure 1: Architecture of HCPM. Upon obtaining coarse feature maps $F_c^A, F_c^B$ from the feature encoder module, they are fed into a self-pruning module for a static ratio top-k selection, denoting the selected feature as $F_{c\_topk}^A, F_{c\_topk}^B$. Subsequently, $F_{c\_topk}^A, F_{c\_topk}^B$ and masks are input into interactive-pruning blocks, which encompass a self-cross attention and a Differentiable Interactive Candidates Selection (DICS) module. The self-cross attention enhances the feature, which then undergoes an automated selection process via the DICS, resulting in pruned candidates. After $N_c$ times iteration, we obtain pruned candidate features $F_{c\_p}^A$ and $F_{c\_p}^B$. Ultimately, the pruned coarse-level features generate a matching matrix and collaborate with fine-level features to regress matching positions, as in LoFTR sun2021loftr.
• Figure 2: Differentiable interactive candidate selection(DICS).
• Figure 3: Two sparse pruning modules explored in HCPM. In implicit-pruning, $\bigoplus$ represents combining the feature and the one-hot mask selected by the DICS module, which are then input into the self-cross (SC) attention module. For direct-pruning, we use $\bigodot$ to input only the selected feature into the self-cross (SC) attention module.
• Figure 4: Iterative-Pruning and Matching Visualization. We have chosen two scenes for visualization. The first column shows the self-pruning result, and the second column displays the interactive-pruning result. Since we supervised with the co-visible area, the visualization may appear inconsistent. The third column presents the matching result.
• Figure 5: Self-Pruning Ratio Analysis.