ISS-Geo142: A Benchmark for Geolocating Astronaut Photography from the International Space Station

TL;DR

This work introduces ISS-Geo142, a benchmark of 142 astronaut images from the International Space Station with ground-truth geographic locations to evaluate geolocation of space-based imagery. It evaluates three baseline pipelines—NN-Geo (VGG16-based convolutional features with AOI cross-correlation), SIFT-Match (SIFT feature matching over a stitched AOI), and TerraByte (GPT-4 Vision-based reasoning combining image content with ISS coordinates)—and provides a standardized evaluation framework. TerraByte achieves the strongest overall performance (~90% accuracy), followed by NN-Geo (~75.5%) and SIFT-Match (high precision in structured regions but high computational cost), highlighting complementary strengths and trade-offs. ISS-Geo142 thus serves as a historically grounded benchmark for future cross-view geolocation in Earth observation, enabling systematic comparison as larger datasets and more capable vision–language models mature.

Abstract

This paper introduces ISS-Geo142, a curated benchmark for geolocating astronaut photography captured from the International Space Station (ISS). Although the ISS position at capture time is known precisely, the specific Earth locations depicted in these images are typically not directly georeferenced, making automated localization non-trivial. ISS-Geo142 consists of 142 images with associated metadata and manually determined geographic locations, spanning a range of spatial scales and scene types. On top of this benchmark, we implement and evaluate three geolocation pipelines: a neural network based approach (NN-Geo) using VGG16 features and cross-correlation over map-derived Areas of Interest (AOIs), a Scale-Invariant Feature Transform based pipeline (SIFT-Match) using sliding-window feature matching on stitched high-resolution AOIs, and TerraByte, an AI system built around a GPT-4 model with vision capabilities that jointly reasons over image content and ISS coordinates. On ISS-Geo142, NN-Geo achieves a match for 75.52\% of the images under our evaluation protocol, SIFT-Match attains high precision on structurally rich scenes at substantial computational cost, and TerraByte establishes the strongest overall baseline, correctly geolocating approximately 90\% of the images while also producing human-readable geographic descriptions. The methods and experiments were originally developed in 2023; this manuscript is a revised and extended version that situates the work relative to subsequent advances in cross-view geo-localization and remote-sensing vision--language models. Taken together, ISS-Geo142 and these three pipelines provide a concrete, historically grounded benchmark for future work on ISS image geolocation.

Figures (7)

• Figure 1: Area range categorization of ISS-Geo142 images. The bar chart shows the distribution of the dataset by surface area coverage on Earth. Area estimates are derived using the field-of-view (FoV) formula, $\text{FoV} = 2 \times \arctan\left(\frac{d}{2f}\right)$, where $d$ is the sensor dimension and $f$ is the focal length of the camera lens. Focal length and camera model are extracted from the image metadata, and sensor dimensions are obtained from camera specifications.
• Figure 2: Geospatial distribution of ISS-Geo142. Blue points indicate the ISS location at the time of capture; red points indicate manually geolocated image footprints. The dataset spans a variety of regions and conditions and serves as a testbed for the three geolocation pipelines.
• Figure 3: Example NN-Geo result. The left image is the query ISS photograph; the right image shows the predicted location within the AOI, marked with a blue dot. In this case, the NN-based algorithm successfully identifies the corresponding region.
• Figure 4: Illustration of the SIFT-based geolocation process. Top left: query ISS image. Right: stitched AOI image from Google Maps tiles. Bottom left: subsampled AOI patch with the highest SIFT matching score, demonstrating successful localization of the query scene.
• Figure 5: Example prompt used with TerraByte: ISS image (left) and associated ISS coordinates (right) provided as context to GPT-4 Vision. The model responds with both a textual description and a candidate geographic location.