Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph-Based Cross-Domain Knowledge Distillation for Cross-Dataset Text-to-Image Person Retrieval

Bingjun Luo, Jinpeng Wang, Wang Zewen, Junjie Zhu, Xibin Zhao

TL;DR

This work tackles cross-dataset text-to-image person retrieval where the target domain lacks labeled cross-modal pairs. It introduces Graph-Based Cross-domain Knowledge Distillation (GCKD), combining a Graph-based Multi-domain Propagation (GMP) module with a Contrastive Momentum Knowledge Distillation (CMKD) module to bridge domain shift and align modalities. Through dynamic cross-domain graphs, embedding memories, and EMA-based teacher-student distillation with cross-modal contrast and fine-grained ITM losses, GCKD achieves superior cross-dataset performance on ICFG-PEDES, RSTPReid, and CUHK-PEDES. The approach demonstrates strong practical potential for unsupervised cross-domain deployment of text-to-image person retrieval systems in surveillance and security contexts.

Abstract

Video surveillance systems are crucial components for ensuring public safety and management in smart city. As a fundamental task in video surveillance, text-to-image person retrieval aims to retrieve the target person from an image gallery that best matches the given text description. Most existing text-to-image person retrieval methods are trained in a supervised manner that requires sufficient labeled data in the target domain. However, it is common in practice that only unlabeled data is available in the target domain due to the difficulty and cost of data annotation, which limits the generalization of existing methods in practical application scenarios. To address this issue, we propose a novel unsupervised domain adaptation method, termed Graph-Based Cross-Domain Knowledge Distillation (GCKD), to learn the cross-modal feature representation for text-to-image person retrieval in a cross-dataset scenario. The proposed GCKD method consists of two main components. Firstly, a graph-based multi-modal propagation module is designed to bridge the cross-domain correlation among the visual and textual samples. Secondly, a contrastive momentum knowledge distillation module is proposed to learn the cross-modal feature representation using the online knowledge distillation strategy. By jointly optimizing the two modules, the proposed method is able to achieve efficient performance for cross-dataset text-to-image person retrieval. acExtensive experiments on three publicly available text-to-image person retrieval datasets demonstrate the effectiveness of the proposed GCKD method, which consistently outperforms the state-of-the-art baselines.

Graph-Based Cross-Domain Knowledge Distillation for Cross-Dataset Text-to-Image Person Retrieval

TL;DR

This work tackles cross-dataset text-to-image person retrieval where the target domain lacks labeled cross-modal pairs. It introduces Graph-Based Cross-domain Knowledge Distillation (GCKD), combining a Graph-based Multi-domain Propagation (GMP) module with a Contrastive Momentum Knowledge Distillation (CMKD) module to bridge domain shift and align modalities. Through dynamic cross-domain graphs, embedding memories, and EMA-based teacher-student distillation with cross-modal contrast and fine-grained ITM losses, GCKD achieves superior cross-dataset performance on ICFG-PEDES, RSTPReid, and CUHK-PEDES. The approach demonstrates strong practical potential for unsupervised cross-domain deployment of text-to-image person retrieval systems in surveillance and security contexts.

Abstract

Video surveillance systems are crucial components for ensuring public safety and management in smart city. As a fundamental task in video surveillance, text-to-image person retrieval aims to retrieve the target person from an image gallery that best matches the given text description. Most existing text-to-image person retrieval methods are trained in a supervised manner that requires sufficient labeled data in the target domain. However, it is common in practice that only unlabeled data is available in the target domain due to the difficulty and cost of data annotation, which limits the generalization of existing methods in practical application scenarios. To address this issue, we propose a novel unsupervised domain adaptation method, termed Graph-Based Cross-Domain Knowledge Distillation (GCKD), to learn the cross-modal feature representation for text-to-image person retrieval in a cross-dataset scenario. The proposed GCKD method consists of two main components. Firstly, a graph-based multi-modal propagation module is designed to bridge the cross-domain correlation among the visual and textual samples. Secondly, a contrastive momentum knowledge distillation module is proposed to learn the cross-modal feature representation using the online knowledge distillation strategy. By jointly optimizing the two modules, the proposed method is able to achieve efficient performance for cross-dataset text-to-image person retrieval. acExtensive experiments on three publicly available text-to-image person retrieval datasets demonstrate the effectiveness of the proposed GCKD method, which consistently outperforms the state-of-the-art baselines.
Paper Structure (25 sections, 9 equations, 2 figures, 4 tables)

This paper contains 25 sections, 9 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text-to-Image Person Retrieval
  4. Unsupervised Domain Adaptation
  5. Problem Definition
  6. Our Model
  7. Model Overview
  8. Graph-based Multi-domain Propagation
  9. Embedding Memory
  10. Graph Construction
  11. GNN Propagation
  12. Contrastive Momentum Knowledge Distillation
  13. Cross-domain Momentum Distillation
  14. Cross-modal Contrastive Learning
  15. Cross-domain Fine-grained Matching
  16. ...and 10 more sections

Figures (2)

  • Figure 1: An illustration of cross-dataset text-to-image person retrieval task. The source dataset contains paired image-text annotations, while the target dataset only contains unpaired images and texts. This task faces both cross-domain shift and cross-modality gap challenges. Part of the figures comes from ding2021semanticallyzhu2021dsslzhu2024improving.
  • Figure 2: The main framework of the proposed Graph-based Cross-domain Knowledge Distillation (GCKD) method. The proposed method consists of two main components: Graph-based Multi-domain Propagation (GMP) module and Contrastive Momentum Knowledge Distillation (CMKD) module.