MATCHA:Towards Matching Anything

TL;DR

MATCHA tackles the long-standing problem of cross-domain correspondences by introducing a unified feature descriptor $F_m$ that enables geometric, semantic, and temporal matching from a single representation. It combines diffusion-based features, semantic cues from DINOv2, and a dynamic transformer-based fusion to leverage complementary knowledge under limited correspondence supervision. The approach yields state-of-the-art results across semantic, geometric, and temporal matching benchmarks and demonstrates the viability of a single, general-purpose descriptor for diverse matching tasks. This unified feature has broad implications for 3D reconstruction, tracking, localization, and image editing, reducing the need for task-specific descriptors.

Abstract

Establishing correspondences across images is a fundamental challenge in computer vision, underpinning tasks like Structure-from-Motion, image editing, and point tracking. Traditional methods are often specialized for specific correspondence types, geometric, semantic, or temporal, whereas humans naturally identify alignments across these domains. Inspired by this flexibility, we propose MATCHA, a unified feature model designed to ``rule them all'', establishing robust correspondences across diverse matching tasks. Building on insights that diffusion model features can encode multiple correspondence types, MATCHA augments this capacity by dynamically fusing high-level semantic and low-level geometric features through an attention-based module, creating expressive, versatile, and robust features. Additionally, MATCHA integrates object-level features from DINOv2 to further boost generalization, enabling a single feature capable of matching anything. Extensive experiments validate that MATCHA consistently surpasses state-of-the-art methods across geometric, semantic, and temporal matching tasks, setting a new foundation for a unified approach for the fundamental correspondence problem in computer vision. To the best of our knowledge, MATCHA is the first approach that is able to effectively tackle diverse matching tasks with a single unified feature.

Figures (17)

• Figure 1: MATCHA for matching anything. We visualize geometric, semantic and temporal correspondences established by MATCHA, using a single feature descriptor.
• Figure 2: Heatmap of features from DINOv2, DIFT, and MATCHA. Given a query point from the source image (1st column), DINOv2 features give more accurate correspondences on single object (1st and 2nd row) but struggle when multiple instances of the same class (3rd row) or similar structures (4th row) exist. Both geometric and semantic features of DIFT perform reversely. By unifying knowledge in the three foundation features, MATCHA produces more accurate and reliable correspondences.
• Figure 3: Architecture of MATCHA. Given an RGB image, MATCHA produces a single feature for geometric, semantic and temporal matching with nearest neighbor searching. MATCHA is built on top of stable diffusion (SD) models rombach2022stablediffusion and DINOv2 oquab2023dinov2. Specifically, original geometric and semantic features extracted from SD are first fused dynamically with a transformer vaswani2017attention consists of self and cross attention blocks. In this dynamic fusion process, both geometric and semantic features are augmented with each other which are supervised with corresponding ground-truth signals in the training process. Then, augmented geometric and semantics features along with DINOv2 feature are unified statically via concatenations into a single feature for matching anything.
• Figure 4: Geometric Matching on HPatches. We report Mean Matching Accuracy (MMA) at error thresholds ranging from 1-10 pixel. Concrete and dash lines denote methods with and without supervision, respectively.
• Figure 5: Visualization of temporal matches on TapVID-Davis doersch2022tapvid. Here we visualize several challenging cases for exstablishing temporal correspondences, where MATCHA generally achieves the best performance in handling extreme scale and viewpoint changes, as well as scenes with multiple similar instances. (DIFT* is the adapted DIFT where we use its concatenated semantic and geometric feature for temporal matching for better performance. )