Adversarial Patch for 3D Local Feature Extractor

TL;DR

This work demonstrates that local feature extractors, exemplified by SuperPoint, can be vulnerable to patch-based adversarial attacks that disrupt matching across viewpoints. It proposes a two-part attack comprising adversarial patch generation and mask generation, using a chessboard baseline and gradient-based optimization within an $8\times8$ cell structure and a $3\times3$ homography $H$ to warp patches across views. Experiments on HPatches show that chessboard and chess-init patches can degrade homography estimation and matching, with scale-aware augmentation improving some robustness and partial transfer to SIFT, though transfer to SuperPoint remains limited. These findings reveal security risks in local feature matching and motivate future work on stronger patterns and defenses.

Abstract

Local feature extractors are the cornerstone of many computer vision tasks. However, their vulnerability to adversarial attacks can significantly compromise their effectiveness. This paper discusses approaches to attack sophisticated local feature extraction algorithms and models to achieve two distinct goals: (1) forcing a match between originally non-matching image regions, and (2) preventing a match between originally matching regions. At the end of the paper, we discuss the performance and drawbacks of different patch generation methods.

Figures (7)

• Figure 1: The model architecture of the SuperPointdetone2018superpoint
• Figure 2: The adversarial patches.
• Figure 3: The generation of the masking for two patches
• Figure 4: The visual result of the matching of a scene from two viewpoints
• Figure 5: The visual results of the scale-invariant experiment with top-150 matching points