Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cluster-Aware Similarity Diffusion for Instance Retrieval

Jifei Luo, Hantao Yao, Changsheng Xu

TL;DR

The paper tackles diffusion-based re-ranking for instance retrieval, where full-graph diffusion can spread misinformation from outliers and inter-manifold samples. It introduces Cluster-Aware Similarity (CAS) diffusion, combining Bidirectional Similarity Diffusion to perform local, symmetric diffusion and Neighbor-guided Similarity Smooth to enforce neighbor-consistency, with an efficient convex formulation and iterative solutions. The framework yields state-of-the-art re-ranking performance on standard benchmarks for both image retrieval and person re-identification, demonstrated through extensive ablations and comparisons with QE-based, diffusion-based, and learning-based methods. The approach offers a robust, scalable alternative to global diffusion by leveraging local clusters and neighborhood structure, with publicly available code for reproducibility and adoption in practice.

Abstract

Diffusion-based re-ranking is a common method used for retrieving instances by performing similarity propagation in a nearest neighbor graph. However, existing techniques that construct the affinity graph based on pairwise instances can lead to the propagation of misinformation from outliers and other manifolds, resulting in inaccurate results. To overcome this issue, we propose a novel Cluster-Aware Similarity (CAS) diffusion for instance retrieval. The primary concept of CAS is to conduct similarity diffusion within local clusters, which can reduce the influence from other manifolds explicitly. To obtain a symmetrical and smooth similarity matrix, our Bidirectional Similarity Diffusion strategy introduces an inverse constraint term to the optimization objective of local cluster diffusion. Additionally, we have optimized a Neighbor-guided Similarity Smoothing approach to ensure similarity consistency among the local neighbors of each instance. Evaluations in instance retrieval and object re-identification validate the effectiveness of the proposed CAS, our code is publicly available.

Cluster-Aware Similarity Diffusion for Instance Retrieval

TL;DR

The paper tackles diffusion-based re-ranking for instance retrieval, where full-graph diffusion can spread misinformation from outliers and inter-manifold samples. It introduces Cluster-Aware Similarity (CAS) diffusion, combining Bidirectional Similarity Diffusion to perform local, symmetric diffusion and Neighbor-guided Similarity Smooth to enforce neighbor-consistency, with an efficient convex formulation and iterative solutions. The framework yields state-of-the-art re-ranking performance on standard benchmarks for both image retrieval and person re-identification, demonstrated through extensive ablations and comparisons with QE-based, diffusion-based, and learning-based methods. The approach offers a robust, scalable alternative to global diffusion by leveraging local clusters and neighborhood structure, with publicly available code for reproducibility and adoption in practice.

Abstract

Diffusion-based re-ranking is a common method used for retrieving instances by performing similarity propagation in a nearest neighbor graph. However, existing techniques that construct the affinity graph based on pairwise instances can lead to the propagation of misinformation from outliers and other manifolds, resulting in inaccurate results. To overcome this issue, we propose a novel Cluster-Aware Similarity (CAS) diffusion for instance retrieval. The primary concept of CAS is to conduct similarity diffusion within local clusters, which can reduce the influence from other manifolds explicitly. To obtain a symmetrical and smooth similarity matrix, our Bidirectional Similarity Diffusion strategy introduces an inverse constraint term to the optimization objective of local cluster diffusion. Additionally, we have optimized a Neighbor-guided Similarity Smoothing approach to ensure similarity consistency among the local neighbors of each instance. Evaluations in instance retrieval and object re-identification validate the effectiveness of the proposed CAS, our code is publicly available.
Paper Structure (20 sections, 8 theorems, 68 equations, 6 figures, 12 tables, 1 algorithm)

This paper contains 20 sections, 8 theorems, 68 equations, 6 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Cluster-Aware Similarity Diffusion
  5. Bidirectional Similarity Diffusion
  6. Neighbor-guided Similarity Smooth
  7. Inference
  8. Experiment
  9. Experiment Setup
  10. Comparison with existing methods
  11. Ablation Study
  12. Conclusion
  13. Bidirectional Similarity Diffusion
  14. Basic Similarity Diffusion Process
  15. Bidirectional Similarity Diffusion Process
  16. ...and 5 more sections

Key Result

Lemma 4.1

Let $\boldsymbol{A}\in\mathbb{R}^{n\times n}$, the spectral radius of $\boldsymbol{A}$ is denoted as $\rho(\boldsymbol{A})=\max\{|\lambda|,\lambda\in\sigma(\boldsymbol{A})\}$, where $\sigma(\boldsymbol{A})$ is the spectrum of $\boldsymbol{A}$ that represents the set of all the eigenvalues. Let $\Ver

Figures (6)

  • Figure 1: The workflow of Cluster-Aware Similarity Diffusion.
  • Figure 2: Explanation of the Neighbor-guided Similarity Smooth strategy. For an instance $x_i$, the average similarity $\boldsymbol{T}_{ij}$ from $x_j$ to its local neighbor is used to constrain the similarity $\boldsymbol{F}_{ij}$. Which can help suppress the influence of instances from different manifolds.
  • Figure 3: Effectiveness of Neighbor-guided Similarity Smooth.
  • Figure 4: Effect of parameter $k_1$, here we plot the sensitivity of a diffusion-based method to its hyper-parameter on the same graph.
  • Figure 5: We showcase the retrieval performance of the initial search and our proposed method through selected qualitative examples, tested on the ROxford dataset. The interest region in the images on the left side are extracted with an orange bounding box as the query. On the right side, the top 10 retrieval results from both the initial search and our proposed method are presented, with true positives marked by a green bounding box and false matches denoted by a red bounding box.
  • ...and 1 more figures

Theorems & Definitions (9)

  • Lemma 4.1
  • Lemma 4.2
  • Lemma 1.1
  • Lemma 1.2
  • Lemma 1.3
  • Lemma 1.4
  • Lemma 1.5
  • Lemma 1.6
  • Definition 1.7