EarthMatch: Iterative Coregistration for Fine-grained Localization of Astronaut Photography

TL;DR

The paper addresses the need for fast, fine-grained geolocalization of astronaut photographs by coupling an EarthLoc-style retrieval step with an iterative coregistration algorithm, EarthMatch. It refines a candidate satellite image through successive homography updates, producing a precise footprint and a confidence measure, with up to four iterations per candidate. A broad benchmark evaluates diverse matchers on a curated AIMS-based dataset, revealing favorable speed-accuracy trade-offs (e.g., RoMa for dense matching, Steerers for fast detector-based matching) and introducing an inlier-thresholding mechanism to suppress false positives. The work also releases a post-retrieval dataset and code to promote reproducibility and accelerate deployment across millions of astronaut images.

Abstract

Precise, pixel-wise geolocalization of astronaut photography is critical to unlocking the potential of this unique type of remotely sensed Earth data, particularly for its use in disaster management and climate change research. Recent works have established the Astronaut Photography Localization task, but have either proved too costly for mass deployment or generated too coarse a localization. Thus, we present EarthMatch, an iterative homography estimation method that produces fine-grained localization of astronaut photographs while maintaining an emphasis on speed. We refocus the astronaut photography benchmark, AIMS, on the geolocalization task itself, and prove our method's efficacy on this dataset. In addition, we offer a new, fair method for image matcher comparison, and an extensive evaluation of different matching models within our localization pipeline. Our method will enable fast and accurate localization of the 4.5 million and growing collection of astronaut photography of Earth. Webpage with code and data at https://earthloc-and-earthmatch.github.io

Figures (8)

• Figure 1: EarthMatch: To produce a pixel-wise geolocalization of an astronaut photograph (the query), we first retrieve a possible candidate from a worldwide database of satellite images. We then compute image correspondences and coregister the two images iteratively, yielding a precise query localization and confidence value.
• Figure 2: Examples of astronaut photo queries from the AIMS dataset Stoken_2023_FMAP which we use in our experiments. The rightmost bottom image is an example of a photo with high tilt/oblique, which we remove prior to our benchmark evaluation. These images are also the least useful/informative for Earth science researchers.
• Figure 3: Left: Overview of retrieval step, which, for a given query, retrieves candidates/predictions from a worldwide database of geo-tagged images. Right: Overview of matching step. The matching pipeline takes as input the query and a retrieved candidate. Surroundings of the candidate are obtained from the database, and then the iterative coregistration (in the form of matching and homographic transformation) is performed.
• Figure 4: Coregistration example. Directly applying the homography to the candidate image results in empty areas (see bottom-right image). Transforming the image along with its surroundings solves this issue (top-right image).
• Figure 5: Examples of images during the EarthMatch procedure. The procedure starts with matching the query $Q$ and the candidate $C_0$ to produce a first homography $H_0$. The candidate surroundings $C_S$ is generated and $H_0$ applied to produce $C_1$. $C_1$ is then matched with $Q$, producing $H_1$ which applied to $C_S$ yields $C_2$. This iterative process continues for a fixed number of iterations (4 iterations in our experiments).