Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Test-time Similarity Modification for Person Re-identification toward Temporal Distribution Shift

Kazuki Adachi, Shohei Enomoto, Taku Sasaki, Shin'ya Yamaguchi

TL;DR

This work tackles the drop in person re-identification performance under temporal distribution shifts by introducing TEMP, the first fully test-time adaptation method for re-id. TEMP operates in an online, open-set setting by redefining uncertainty through re-id entropy computed from query-gallery similarities, updating only BN affine parameters to refine similarities without storing source data. Empirical results on Market-1501, MSMT17, and PersonX show TEMP substantially improves top-1 accuracy under location changes and image corruptions (up to ~9 points) and aligns query-gallery feature distributions in the learned space. The approach is lightweight, avoids source-data retention, and can be extended with additional anti-forgetting strategies for further gains.

Abstract

Person re-identification (re-id), which aims to retrieve images of the same person in a given image from a database, is one of the most practical image recognition applications. In the real world, however, the environments that the images are taken from change over time. This causes a distribution shift between training and testing and degrades the performance of re-id. To maintain re-id performance, models should continue adapting to the test environment's temporal changes. Test-time adaptation (TTA), which aims to adapt models to the test environment with only unlabeled test data, is a promising way to handle this problem because TTA can adapt models instantly in the test environment. However, the previous TTA methods are designed for classification and cannot be directly applied to re-id. This is because the set of people's identities in the dataset differs between training and testing in re-id, whereas the set of classes is fixed in the current TTA methods designed for classification. To improve re-id performance in changing test environments, we propose TEst-time similarity Modification for Person re-identification (TEMP), a novel TTA method for re-id. TEMP is the first fully TTA method for re-id, which does not require any modification to pre-training. Inspired by TTA methods that refine the prediction uncertainty in classification, we aim to refine the uncertainty in re-id. However, the uncertainty cannot be computed in the same way as classification in re-id since it is an open-set task, which does not share person labels between training and testing. Hence, we propose re-id entropy, an alternative uncertainty measure for re-id computed based on the similarity between the feature vectors. Experiments show that the re-id entropy can measure the uncertainty on re-id and TEMP improves the performance of re-id in online settings where the distribution changes over time.

Test-time Similarity Modification for Person Re-identification toward Temporal Distribution Shift

TL;DR

This work tackles the drop in person re-identification performance under temporal distribution shifts by introducing TEMP, the first fully test-time adaptation method for re-id. TEMP operates in an online, open-set setting by redefining uncertainty through re-id entropy computed from query-gallery similarities, updating only BN affine parameters to refine similarities without storing source data. Empirical results on Market-1501, MSMT17, and PersonX show TEMP substantially improves top-1 accuracy under location changes and image corruptions (up to ~9 points) and aligns query-gallery feature distributions in the learned space. The approach is lightweight, avoids source-data retention, and can be extended with additional anti-forgetting strategies for further gains.

Abstract

Person re-identification (re-id), which aims to retrieve images of the same person in a given image from a database, is one of the most practical image recognition applications. In the real world, however, the environments that the images are taken from change over time. This causes a distribution shift between training and testing and degrades the performance of re-id. To maintain re-id performance, models should continue adapting to the test environment's temporal changes. Test-time adaptation (TTA), which aims to adapt models to the test environment with only unlabeled test data, is a promising way to handle this problem because TTA can adapt models instantly in the test environment. However, the previous TTA methods are designed for classification and cannot be directly applied to re-id. This is because the set of people's identities in the dataset differs between training and testing in re-id, whereas the set of classes is fixed in the current TTA methods designed for classification. To improve re-id performance in changing test environments, we propose TEst-time similarity Modification for Person re-identification (TEMP), a novel TTA method for re-id. TEMP is the first fully TTA method for re-id, which does not require any modification to pre-training. Inspired by TTA methods that refine the prediction uncertainty in classification, we aim to refine the uncertainty in re-id. However, the uncertainty cannot be computed in the same way as classification in re-id since it is an open-set task, which does not share person labels between training and testing. Hence, we propose re-id entropy, an alternative uncertainty measure for re-id computed based on the similarity between the feature vectors. Experiments show that the re-id entropy can measure the uncertainty on re-id and TEMP improves the performance of re-id in online settings where the distribution changes over time.
Paper Structure (22 sections, 5 equations, 5 figures, 3 tables, 1 algorithm)

This paper contains 22 sections, 5 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Unsupervised Domain Adaptation for Person Re-identification
  4. Test-time Adaptation
  5. Problem Setting
  6. Person Re-identification
  7. Fully Test-time Adaptation in Person Re-identification
  8. Test-time Similarity Modification for Person Re-identification
  9. Experiment
  10. Experimental Setting
  11. Dataset
  12. Pre-training of the Re-id Model
  13. Test-time Adaptation
  14. Results
  15. Distribution Shift and Re-id Entropy
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Overview of TEMP. TEMP alleviates the drop in performance of a re-id model caused by a distribution shift by using re-id entropy to modify the similarities between the query and gallery features. The person images in the figure are sampled from PersonX Sun_2019_CVPR.
  • Figure 2: Top-1 CMC accuracy, mean re-id entropy, and corruption strength on Market-1501. As the corruption strength (represented by the color of the circle, darker is stronger) increases, the re-id entropy also increases, and the top-1 CMC accuracy deteriorates.
  • Figure 3: Top-1 CMC accuracy vs. iteration. The exponential moving average of the CMCs computed for each mini-batch is plotted. (a): The location change scenario. The source datasets used to plot the top, middle, and bottom rows are Market-1501, MSMT17, and PersonX, respectively. The vertical dashed lines represent the iterations when the dataset switches. MA, MS, and PX in the figure represent Market-1501, MSMT17, and PersonX. (b): The corruption scenario. The source dataset is Market-1501. The top, middle, and bottom rows correspond to brightness, Gaussian blur, and pixelate corruptions. The vertical dashed lines represent the iterations when the strengths of the corruptions change. BNTA han2022generalizable is not plotted since it immediately decreased the accuracy to near to zero.
  • Figure 4: Sensitivity analysis of $k$.
  • Figure 5: UMAP mcinnes2018umap-software visualization of the query and gallery features in the corruption scenario. The gallery images are not corrupted, and the query ones are Gaussian-blurred. TEMP aligns the distribution of the gallery (blue dots) and query (orange dots) features.