Multiple Code Hashing for Efficient Image Retrieval

TL;DR

This work tackles the inefficiency of hash bucket search in large-scale image retrieval caused by representing each image with a single hash code. It introduces Multiple Code Hashing (MCH), a framework that learns multiple region-specific binary codes per image and employs a deep reinforcement learning agent to decide which region codes to keep, enabling tighter coupling of similar images in the Hamming space. The learning process is decomposed into a base hashing model learning step and an agent learning step using REINFORCE, with an out-of-sample extension for unseen data. Empirical results on NUS-WIDE, MS-COCO, and MIR Flickr show that MCH improves recall and precision within the Hamming ball, reduces the number of visited hash buckets, and maintains or enhances mean average precision compared with single-code baselines. Overall, MCH offers a practical, scalable path to faster image retrieval by leveraging region-aware multi-code representations and reinforcement learning for code selection.

Abstract

Due to its low storage cost and fast query speed, hashing has been widely used in large-scale image retrieval tasks. Hash bucket search returns data points within a given Hamming radius to each query, which can enable search at a constant or sub-linear time cost. However, existing hashing methods cannot achieve satisfactory retrieval performance for hash bucket search in complex scenarios, since they learn only one hash code for each image. More specifically, by using one hash code to represent one image, existing methods might fail to put similar image pairs to the buckets with a small Hamming distance to the query when the semantic information of images is complex. As a result, a large number of hash buckets need to be visited for retrieving similar images, based on the learned codes. This will deteriorate the efficiency of hash bucket search. In this paper, we propose a novel hashing framework, called multiple code hashing (MCH), to improve the performance of hash bucket search. The main idea of MCH is to learn multiple hash codes for each image, with each code representing a different region of the image. Furthermore, we propose a deep reinforcement learning algorithm to learn the parameters in MCH. To the best of our knowledge, this is the first work that proposes to learn multiple hash codes for each image in image retrieval. Experiments demonstrate that MCH can achieve a significant improvement in hash bucket search, compared with existing methods that learn only one hash code for each image.

Figures (9)

• Figure 1: An example for illustrating the shortcoming of existing hashing methods.
• Figure 2: An illustration of our MCH framework, which consists of two steps. The first step is called base hashing model learning step, which learns a hash function through similarity-preserving learning. The second step is called agent learning step, which learns a decision strategy about multiple hash codes for image ${\bf x}_i$ through exploring different regions ${\bf x}_i^*$ that can better preserve the pairwise similarity than the whole image. Best viewed in color.
• Figure 3: Recall within Hamming radius $0$ on the three benchmark datasets.
• Figure 4: Precision within Hamming radius $0$ on the three benchmark datasets.
• Figure 5: The F1-bucket curves on the three benchmark datasets with 16 bits.