Annotation Cost-Efficient Active Learning for Deep Metric Learning Driven Remote Sensing Image Retrieval
Genc Hoxha, Gencer Sumbul, Julia Henkel, Lars Möllenbrok, Begüm Demir
TL;DR
This work tackles the high annotation cost of deep metric learning for remote-sensing CBIR by proposing ANNEAL, an annotation cost-efficient active learning framework that builds a compact, informative training set of similar/dissimilar image pairs. ANNEAL alternates between selecting the most uncertain image pairs and enforcing diversity among them, using either metric-guided uncertainty estimation (MGUE) or binary classifier guided uncertainty estimation (BCGUE), followed by a clustering-based diversity step. The method demonstrates superior CBIR performance and faster convergence than random or classification-based AL on UC-Merced and AID benchmarks, while reducing labeling needs to a single bit per pair and enabling transitive expansion of the training set. The approach constitutes a practical, scalable solution for RS CBIR with DML, and the authors release code for public use.
Abstract
Deep metric learning (DML) has shown to be effective for content-based image retrieval (CBIR) in remote sensing (RS). Most of DML methods for CBIR rely on a high number of annotated images to accurately learn model parameters of deep neural networks (DNNs). However, gathering such data is time-consuming and costly. To address this, we propose an annotation cost-efficient active learning (ANNEAL) method tailored to DML-driven CBIR in RS. ANNEAL aims to create a small but informative training set made up of similar and dissimilar image pairs to be utilized for accurately learning a metric space. The informativeness of image pairs is evaluated by combining uncertainty and diversity criteria. To assess the uncertainty of image pairs, we introduce two algorithms: 1) metric-guided uncertainty estimation (MGUE); and 2) binary classifier guided uncertainty estimation (BCGUE). MGUE algorithm automatically estimates a threshold value that acts as a boundary between similar and dissimilar image pairs based on the distances in the metric space. The closer the similarity between image pairs is to the estimated threshold value the higher their uncertainty. BCGUE algorithm estimates the uncertainty of the image pairs based on the confidence of the classifier in assigning correct similarity labels. The diversity criterion is assessed through a clustering-based strategy. ANNEAL combines either MGUE or BCGUE algorithm with the clustering-based strategy to select the most informative image pairs, which are then labelled by expert annotators as similar or dissimilar. This way of annotating images significantly reduces the annotation cost compared to annotating images with land-use land-cover class labels. Experimental results on two RS benchmark datasets demonstrate the effectiveness of our method. The code of this work is publicly available at \url{https://git.tu-berlin.de/rsim/anneal_tgrs}.