Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval

TL;DR

The paper presents Google Landmarks Dataset v2 (GLDv2), the largest-scale benchmark for instance-level landmark recognition and retrieval, featuring over 5.0M images and 200k landmarks drawn from freely licensed Wikimedia Commons images. It emphasizes real-world challenges such as extreme long-tailed class distributions, substantial intra-class variability, and a test set rich in out-of-domain queries, and provides ground-truth ground truth enhanced by a large-scale, multi-rater re-annotation effort. The authors establish clear evaluation metrics ($\mu$AP for recognition and $\mathrm{mAP@100}$ for retrieval) and demonstrate the transfer-learning potential by training descriptors on GLDv2 and evaluating on independent datasets, with baseline methods spanning global descriptors and local-feature re-ranking. The dataset, ground-truth, and scoring code are publicly available, aiming to spur robust, scalable, and transferable instance-level recognition and retrieval technologies applicable to real-world visual search and recognition tasks.

Abstract

While image retrieval and instance recognition techniques are progressing rapidly, there is a need for challenging datasets to accurately measure their performance -- while posing novel challenges that are relevant for practical applications. We introduce the Google Landmarks Dataset v2 (GLDv2), a new benchmark for large-scale, fine-grained instance recognition and image retrieval in the domain of human-made and natural landmarks. GLDv2 is the largest such dataset to date by a large margin, including over 5M images and 200k distinct instance labels. Its test set consists of 118k images with ground truth annotations for both the retrieval and recognition tasks. The ground truth construction involved over 800 hours of human annotator work. Our new dataset has several challenging properties inspired by real world applications that previous datasets did not consider: An extremely long-tailed class distribution, a large fraction of out-of-domain test photos and large intra-class variability. The dataset is sourced from Wikimedia Commons, the world's largest crowdsourced collection of landmark photos. We provide baseline results for both recognition and retrieval tasks based on state-of-the-art methods as well as competitive results from a public challenge. We further demonstrate the suitability of the dataset for transfer learning by showing that image embeddings trained on it achieve competitive retrieval performance on independent datasets. The dataset images, ground-truth and metric scoring code are available at https://github.com/cvdfoundation/google-landmark.

Figures (12)

• Figure 1: The Google Landmarks Dataset v2 contains a variety of natural and human-made landmarks from around the world. Since the class distribution is very long-tailed, the dataset contains a large number of lesser-known local landmarks.
• Figure 2: Heatmap of the places in the Google Landmarks Dataset v2.
• Figure 3: Histogram of the number of images from the top-20 countries (blue) compared to their populations (red).
• Figure 4: Histogram of the number of images per continent (blue) compared to their populations (red).
• Figure 5: Histogram of the number of images by landmark category. This includes only categories with more than 25.0k images.