Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Alice Benchmarks: Connecting Real World Re-Identification with the Synthetic

Xiaoxiao Sun, Yue Yao, Shengjin Wang, Hongdong Li, Liang Zheng

TL;DR

The settings of Alice benchmarks are detail, an analysis of existing commonly-used domain adaptation methods are provided, and an online server has been set up for the community to evaluate methods conveniently and fairly.

Abstract

For object re-identification (re-ID), learning from synthetic data has become a promising strategy to cheaply acquire large-scale annotated datasets and effective models, with few privacy concerns. Many interesting research problems arise from this strategy, e.g., how to reduce the domain gap between synthetic source and real-world target. To facilitate developing more new approaches in learning from synthetic data, we introduce the Alice benchmarks, large-scale datasets providing benchmarks as well as evaluation protocols to the research community. Within the Alice benchmarks, two object re-ID tasks are offered: person and vehicle re-ID. We collected and annotated two challenging real-world target datasets: AlicePerson and AliceVehicle, captured under various illuminations, image resolutions, etc. As an important feature of our real target, the clusterability of its training set is not manually guaranteed to make it closer to a real domain adaptation test scenario. Correspondingly, we reuse existing PersonX and VehicleX as synthetic source domains. The primary goal is to train models from synthetic data that can work effectively in the real world. In this paper, we detail the settings of Alice benchmarks, provide an analysis of existing commonly-used domain adaptation methods, and discuss some interesting future directions. An online server has been set up for the community to evaluate methods conveniently and fairly. Datasets and the online server details are available at https://sites.google.com/view/alice-benchmarks.

Alice Benchmarks: Connecting Real World Re-Identification with the Synthetic

TL;DR

The settings of Alice benchmarks are detail, an analysis of existing commonly-used domain adaptation methods are provided, and an online server has been set up for the community to evaluate methods conveniently and fairly.

Abstract

For object re-identification (re-ID), learning from synthetic data has become a promising strategy to cheaply acquire large-scale annotated datasets and effective models, with few privacy concerns. Many interesting research problems arise from this strategy, e.g., how to reduce the domain gap between synthetic source and real-world target. To facilitate developing more new approaches in learning from synthetic data, we introduce the Alice benchmarks, large-scale datasets providing benchmarks as well as evaluation protocols to the research community. Within the Alice benchmarks, two object re-ID tasks are offered: person and vehicle re-ID. We collected and annotated two challenging real-world target datasets: AlicePerson and AliceVehicle, captured under various illuminations, image resolutions, etc. As an important feature of our real target, the clusterability of its training set is not manually guaranteed to make it closer to a real domain adaptation test scenario. Correspondingly, we reuse existing PersonX and VehicleX as synthetic source domains. The primary goal is to train models from synthetic data that can work effectively in the real world. In this paper, we detail the settings of Alice benchmarks, provide an analysis of existing commonly-used domain adaptation methods, and discuss some interesting future directions. An online server has been set up for the community to evaluate methods conveniently and fairly. Datasets and the online server details are available at https://sites.google.com/view/alice-benchmarks.
Paper Structure (17 sections, 6 figures, 6 tables)

This paper contains 17 sections, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Synthetic Data in Alice
  4. Real-world Data in Alice
  5. Evaluation Protocols
  6. Baseline Evaluation
  7. Method Description
  8. Effectiveness of Content-level DA Method
  9. Effectiveness of Traditional DA Method
  10. Visualization of DA Methods
  11. Discussion and Future Work
  12. Conclusion
  13. Summary of Existing Synthetic Datasets
  14. Data Annotation
  15. Links to datasets
  16. ...and 2 more sections

Figures (6)

  • Figure 1: The current version of Alice benchmarks supports two research tasks: person and vehicle re-ID. The source domains are synthetic images from the PersonX and VehicleX datasets. The data of the target domains are real-world images we collected from varying conditions.
  • Figure 2: Camera topology in AliceVehicle. (A): Geological positions of cameras. (B): Camera field of views.
  • Figure 3: Performance of domain adaptation methods from synthetic to real-world datasets. A: person re-ID using PersonX as the source. B: vehicle re-ID using VehicleX as the source. Under each target dataset, we use light green to indicate the accuracy of various methods and dark green to show the improvement brought by attribute descent combined with these methods.
  • Figure 4: Sample images on person (left) and vehicle re-ID (right) before and after content/pixel domain adaptation. There are source images generated by (A1-2) random attributes, (B1-2) attribute descent, (C1-2) SPGAN and (D1-2) SPGAN $\&$ attribute descent. E1-2: show samples from the target domain. We find that attribute descent changes the viewpoint and illuminations etc., visual factors, of the objects, while SPGAN increases image blurring and adapts the image color to the target style.
  • Figure 5: Graphical user interface (GUI) used in our annotation process. AlicePerson (Top) and AliceVehicle (Bottom). Annotators work together according to the time of video recording. Each annotator is in charge of 1-2 cameras, looking for the same ID together.
  • ...and 1 more figures