Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CM-Bench: A Comprehensive Cross-Modal Feature Matching Benchmark Bridging Visible and Infrared Images

Liangzheng Sun, Mengfan He, Xingyu Shao, Binbin Li, Zhiqiang Yan, Chunyu Li, Ziyang Meng, Fei Xing

Abstract

Infrared-visible (IR-VIS) feature matching plays an essential role in cross-modality visual localization, navigation and perception. Along with the rapid development of deep learning techniques, a number of representative image matching methods have been proposed. However, crossmodal feature matching is still a challenging task due to the significant appearance difference. A significant gap for cross-modal feature matching research lies in the absence of standardized benchmarks and metrics for evaluations. In this paper, we introduce a comprehensive cross-modal feature matching benchmark, CM-Bench, which encompasses 30 feature matching algorithms across diverse cross-modal datasets. Specifically, state-of-the-art traditional and deep learning-based methods are first summarized and categorized into sparse, semidense, and dense methods. These methods are evaluated by different tasks including homography estimation, relative pose estimation, and feature-matching-based geo-localization. In addition, we introduce a classification-network-based adaptive preprocessing front-end that automatically selects suitable enhancement strategies before matching. We also present a novel infrared-satellite cross-modal dataset with manually annotated ground-truth correspondences for practical geo-localization evaluation. The dataset and resource will be available at: https://github.com/SLZ98/CM-Bench.

CM-Bench: A Comprehensive Cross-Modal Feature Matching Benchmark Bridging Visible and Infrared Images

Abstract

Infrared-visible (IR-VIS) feature matching plays an essential role in cross-modality visual localization, navigation and perception. Along with the rapid development of deep learning techniques, a number of representative image matching methods have been proposed. However, crossmodal feature matching is still a challenging task due to the significant appearance difference. A significant gap for cross-modal feature matching research lies in the absence of standardized benchmarks and metrics for evaluations. In this paper, we introduce a comprehensive cross-modal feature matching benchmark, CM-Bench, which encompasses 30 feature matching algorithms across diverse cross-modal datasets. Specifically, state-of-the-art traditional and deep learning-based methods are first summarized and categorized into sparse, semidense, and dense methods. These methods are evaluated by different tasks including homography estimation, relative pose estimation, and feature-matching-based geo-localization. In addition, we introduce a classification-network-based adaptive preprocessing front-end that automatically selects suitable enhancement strategies before matching. We also present a novel infrared-satellite cross-modal dataset with manually annotated ground-truth correspondences for practical geo-localization evaluation. The dataset and resource will be available at: https://github.com/SLZ98/CM-Bench.
Paper Structure (22 sections, 3 equations, 3 figures, 5 tables)

This paper contains 22 sections, 3 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. RGB-Only Image Feature Matching
  4. Sparse Keypoint-Based Matching
  5. Semi-Dense Correspondence
  6. Dense Correspondence
  7. Cross-Modal Feature Matching
  8. Methodology
  9. Cross-Modal Feature Matching and Datasets
  10. Adaptive Preprocessing Front-end
  11. Evaluation Methodology and Metrics
  12. Homography Estimation Metrics and Relative Pose Estimation Metrics
  13. Geo-localization Metrics
  14. Experiments and Results
  15. Experimental Setup
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Overview of eight representative matchers across four benchmark tasks, together with representative samples from the corresponding evaluation settings.
  • Figure 2: Overview of the proposed ThermoSat dataset. (A) Collection area overview with seven representative thermal-to-satellite localization regions. (B) Representative ThermoSat scenes. (C) Cross-modal ground-truth correspondences, where blue dots denote satellite ground-truth points, red crosses denote thermal ground-truth points, and green lines denote ground-truth correspondences.
  • Figure 3: Overview of the proposed adaptive preprocessing front-end. A lightweight MobileNetV4-based classifier predicts the most suitable preprocessing branch for each IR-VIS image pair before feature matching.