Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Domain Generalizable Person Search Using Unreal Dataset

Minyoung Oh, Duhyun Kim, Jae-Young Sim

TL;DR

This work tackles the high cost and privacy concerns of collecting labeled real data for person search by proposing a domain generalization framework trained solely on an automatically labeled unreal dataset. It introduces fidelity adaptive training to estimate and leverage instance fidelity, weighting detection, confidence, and feature updates to bridge unreal-to-real domain gaps. A domain invariant feature learning scheme, treating each unreal sequence as a separate domain, further suppresses domain-specific cues via domain-guided normalization, a domain separation loss, and domain feature updates. The approach achieves competitive results with existing supervised, weakly supervised, and unsupervised domain-adaptation methods on unseen real datasets, demonstrating practical, labeling-free generalization for real-world deployment.

Abstract

Collecting and labeling real datasets to train the person search networks not only requires a lot of time and effort, but also accompanies privacy issues. The weakly-supervised and unsupervised domain adaptation methods have been proposed to alleviate the labeling burden for target datasets, however, their generalization capability is limited. We introduce a novel person search method based on the domain generalization framework, that uses an automatically labeled unreal dataset only for training but is applicable to arbitrary unseen real datasets. To alleviate the domain gaps when transferring the knowledge from the unreal source dataset to the real target datasets, we estimate the fidelity of person instances which is then used to train the end-to-end network adaptively. Moreover, we devise a domain-invariant feature learning scheme to encourage the network to suppress the domain-related features. Experimental results demonstrate that the proposed method provides the competitive performance to existing person search methods even though it is applicable to arbitrary unseen datasets without any prior knowledge and re-training burdens.

Domain Generalizable Person Search Using Unreal Dataset

TL;DR

This work tackles the high cost and privacy concerns of collecting labeled real data for person search by proposing a domain generalization framework trained solely on an automatically labeled unreal dataset. It introduces fidelity adaptive training to estimate and leverage instance fidelity, weighting detection, confidence, and feature updates to bridge unreal-to-real domain gaps. A domain invariant feature learning scheme, treating each unreal sequence as a separate domain, further suppresses domain-specific cues via domain-guided normalization, a domain separation loss, and domain feature updates. The approach achieves competitive results with existing supervised, weakly supervised, and unsupervised domain-adaptation methods on unseen real datasets, demonstrating practical, labeling-free generalization for real-world deployment.

Abstract

Collecting and labeling real datasets to train the person search networks not only requires a lot of time and effort, but also accompanies privacy issues. The weakly-supervised and unsupervised domain adaptation methods have been proposed to alleviate the labeling burden for target datasets, however, their generalization capability is limited. We introduce a novel person search method based on the domain generalization framework, that uses an automatically labeled unreal dataset only for training but is applicable to arbitrary unseen real datasets. To alleviate the domain gaps when transferring the knowledge from the unreal source dataset to the real target datasets, we estimate the fidelity of person instances which is then used to train the end-to-end network adaptively. Moreover, we devise a domain-invariant feature learning scheme to encourage the network to suppress the domain-related features. Experimental results demonstrate that the proposed method provides the competitive performance to existing person search methods even though it is applicable to arbitrary unseen datasets without any prior knowledge and re-training burdens.
Paper Structure (27 sections, 8 equations, 7 figures, 8 tables)

This paper contains 27 sections, 8 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Person Search
  4. Domain Generalization
  5. Unreal Dataset
  6. Method
  7. Fidelity Adaptive Training
  8. Fidelity Estimation.
  9. Fidelity Weighted Detection Loss.
  10. Fidelity Guided Confidence Loss.
  11. Fidelity Weighted Feature Update.
  12. Domain Invariant Feature Learning
  13. Domain-Guided Feature Normalization.
  14. Domain Separation Loss.
  15. Domain Feature Update.
  16. ...and 12 more sections

Figures (7)

  • Figure 1: The proposed domain generalization concept compared to the weakly supervised and unsupervised domain adaptation methods. The upper and lower figures represent the training datasets and the test datasets, respectively.
  • Figure 2: The characteristics of the real PRW (left) and unreal JTA (right) datasets. The identity labels of persons are shown at the top of the bounding boxes.
  • Figure 3: Images from the unreal JTA dataset.
  • Figure 4: The overall framework of the proposed method. At the training phase, the ID-specific and domain-specific features are extracted by using the attention encoders where the ID-specific features are used to estimate the fidelity of person instance. The estimated fidelity is then used to adaptively compute $\mathcal{L}_\mathrm{det}$ and $\mathcal{L}_\mathrm{con}$ in the head network. The domain-specific features are used to calculate $\mathcal{L}_\mathrm{dom}$ and $\mathcal{L}_\mathrm{sep}$. At the inference phase, only the ID-specific features are used. The dashed lines indicate the stop-gradient operation.
  • Figure 5: The BRISQUE score distributions for the cropped images of person instances.
  • ...and 2 more figures