Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Streamlined Attention-Based Network for Descriptor Extraction

Mattia D'Urso, Emanuele Santellani, Christian Sormann, Mattia Rossi, Andreas Kuhn, Friedrich Fraundorfer

TL;DR

SANDesc tackles the need for robust, scalable descriptors that can be paired with existing keypoint detectors without retraining detectors. It introduces a lightweight Residual U‑Net Block with Attention (RUBA) to produce a 128‑channel per-pixel descriptor, trained with a modified triplet loss and curriculum hard negative mining, achieving strong performance with only 2.4M parameters. Extensive evaluations on HPatches, MegaDepth‑1500, IMC 2021, and Graz4K show consistent improvements in matching reliability and geometric accuracy, while maintaining efficiency and scalability for high‑resolution imagery. The work also provides a new 4K urban Graz4K benchmark, highlighting practical gains for large‑scale SfM and localization pipelines and demonstrating competitive performance against larger decoupled models such as DeDoDe‑G.

Abstract

We introduce SANDesc, a Streamlined Attention-Based Network for Descriptor extraction that aims to improve on existing architectures for keypoint description. Our descriptor network learns to compute descriptors that improve matching without modifying the underlying keypoint detector. We employ a revised U-Net-like architecture enhanced with Convolutional Block Attention Modules and residual paths, enabling effective local representation while maintaining computational efficiency. We refer to the building blocks of our model as Residual U-Net Blocks with Attention. The model is trained using a modified triplet loss in combination with a curriculum learning-inspired hard negative mining strategy, which improves training stability. Extensive experiments on HPatches, MegaDepth-1500, and the Image Matching Challenge 2021 show that training SANDesc on top of existing keypoint detectors leads to improved results on multiple matching tasks compared to the original keypoint descriptors. At the same time, SANDesc has a model complexity of just 2.4 million parameters. As a further contribution, we introduce a new urban dataset featuring 4K images and pre-calibrated intrinsics, designed to evaluate feature extractors. On this benchmark, SANDesc achieves substantial performance gains over the existing descriptors while operating with limited computational resources.

A Streamlined Attention-Based Network for Descriptor Extraction

TL;DR

SANDesc tackles the need for robust, scalable descriptors that can be paired with existing keypoint detectors without retraining detectors. It introduces a lightweight Residual U‑Net Block with Attention (RUBA) to produce a 128‑channel per-pixel descriptor, trained with a modified triplet loss and curriculum hard negative mining, achieving strong performance with only 2.4M parameters. Extensive evaluations on HPatches, MegaDepth‑1500, IMC 2021, and Graz4K show consistent improvements in matching reliability and geometric accuracy, while maintaining efficiency and scalability for high‑resolution imagery. The work also provides a new 4K urban Graz4K benchmark, highlighting practical gains for large‑scale SfM and localization pipelines and demonstrating competitive performance against larger decoupled models such as DeDoDe‑G.

Abstract

We introduce SANDesc, a Streamlined Attention-Based Network for Descriptor extraction that aims to improve on existing architectures for keypoint description. Our descriptor network learns to compute descriptors that improve matching without modifying the underlying keypoint detector. We employ a revised U-Net-like architecture enhanced with Convolutional Block Attention Modules and residual paths, enabling effective local representation while maintaining computational efficiency. We refer to the building blocks of our model as Residual U-Net Blocks with Attention. The model is trained using a modified triplet loss in combination with a curriculum learning-inspired hard negative mining strategy, which improves training stability. Extensive experiments on HPatches, MegaDepth-1500, and the Image Matching Challenge 2021 show that training SANDesc on top of existing keypoint detectors leads to improved results on multiple matching tasks compared to the original keypoint descriptors. At the same time, SANDesc has a model complexity of just 2.4 million parameters. As a further contribution, we introduce a new urban dataset featuring 4K images and pre-calibrated intrinsics, designed to evaluate feature extractors. On this benchmark, SANDesc achieves substantial performance gains over the existing descriptors while operating with limited computational resources.
Paper Structure (22 sections, 2 equations, 4 figures, 7 tables)

This paper contains 22 sections, 2 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learned Patch Descriptors.
  4. Joint Detection and Description.
  5. The Rationale for Decoupling.
  6. Modern Decoupled Architectures.
  7. Method
  8. Residual U-Net Block with Attention
  9. Loss
  10. Training Setup
  11. Dataset
  12. Details
  13. Experiments
  14. HPatches
  15. The Image Matching Challenge 2021
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Qualitative comparison. Examples of local feature matching between pairs of images exhibiting large scale differences. Inliers are displayed in green; outliers in red.
  • Figure 2: Qualitative comparison. Examples of local feature matching between pairs of images exhibiting large scale differences. Inliers are displayed in green; outliers in red.
  • Figure 3: Qualitative comparison. Example of local feature matching between a pair of images exhibiting large scale differences. Inliers are displayed in green; outliers in red.
  • Figure 4: Residual U-Net Block with Attention (RUBA). This block serves as the fundamental building unit of the four-layer U-Net. We use four blocks in the encoder and four in the decoder, configured to resize the features through pooling and upsampling, respectively.