Anti-Forgetting Adaptation for Unsupervised Person Re-identification

TL;DR

This work tackles forgetting, poor cross-domain generalization, and backward-compatibility in unsupervised lifelong person ReID. It introduces DJAA, a Dual-level Joint Adaptation and Anti-forgetting framework that combines an Adaptation Module with prototype- and instance-level contrastive losses ($\mathcal{L}_{pa}$, $\mathcal{L}_{ia}$) and a Rehearsal Module that enforces image-to-prototype and image-to-image consistency ($\mathcal{L}_{ps}$, $\mathcal{L}_{is}$) via a hybrid memory buffer of representative images and cluster prototypes. The method achieves strong anti-forgetting on seen domains, superior generalization to unseen domains, and backward-compatible representations across multiple adaptation steps, outperforming state-of-the-art baselines such as CVS and CLUDA. Key contributions include a clustering-guided memory buffer update strategy, dual-level contrastive learning for domain adaptation, and parity between old and new domain knowledge through consistency regularization. The approach offers practical benefits for real-world lifelong ReID deployments by reducing re-extraction of old gallery features and enabling robust cross-domain retrieval.

Abstract

Regular unsupervised domain adaptive person re-identification (ReID) focuses on adapting a model from a source domain to a fixed target domain. However, an adapted ReID model can hardly retain previously-acquired knowledge and generalize to unseen data. In this paper, we propose a Dual-level Joint Adaptation and Anti-forgetting (DJAA) framework, which incrementally adapts a model to new domains without forgetting source domain and each adapted target domain. We explore the possibility of using prototype and instance-level consistency to mitigate the forgetting during the adaptation. Specifically, we store a small number of representative image samples and corresponding cluster prototypes in a memory buffer, which is updated at each adaptation step. With the buffered images and prototypes, we regularize the image-to-image similarity and image-to-prototype similarity to rehearse old knowledge. After the multi-step adaptation, the model is tested on all seen domains and several unseen domains to validate the generalization ability of our method. Extensive experiments demonstrate that our proposed method significantly improves the anti-forgetting, generalization and backward-compatible ability of an unsupervised person ReID model.

Figures (6)

• Figure 1: ReID feature space comparison vandermaaten08a of regular unsupervised ReID method ICE Chen_2021_ICE and unsupervised lifelong ReID method DJAA on three scenarios. 1) Seen domain non-forgetting ability: DJAA preserves previously acquired knowledge, which reduces the gap in the feature space between seen domains. 2) Unseen domain generalization ability: DJAA accumulates domain-shared features that reduce the gap between unseen domains. 3) Backward-compatible ability: With DJAA, updated query (Query_new) and gallery (Gallery_new) representations remain in the same feature space with the previously extracted representations (Query_old, Gallery_old). Query_old and Gallery_old are Market1501 representations extracted after one adaptation step, while Query_new and Gallery_new are Market1501 representations extracted after three adaptation steps.
• Figure 2: (a) General architecture of DJAA: our proposed framework is composed of an adaptation module and a rehearsal module. A memory buffer stores a small number of images and corresponding cluster prototypes from previous step $s-1$, which are trained jointly with data from step $s$. The two modules work collaboratively to achieve the adaptation without forgetting. (b) Adaptation Module: This module follows the new domain pathway ($\rightarrow$) to learn new domain knowledge with dual-level contrastive adaptation losses $\mathcal{L}_{pa}$ and $\mathcal{L}_{ia}$. (c) Rehearsal Module: This module follows the old domain rehearsal pathway($\rightarrow$) to rehearse old domain knowledge with dual-level similarity consistency losses $\mathcal{L}_{ps}$ and $\mathcal{L}_{is}$.
• Figure 3: We introduce two kinds of perturbations, including domain gap from different encoders and data augmentation. Our target is to make the model invariant to domain gap and data augmentation perturbations.
• Figure 4: Non-forgetting evaluation with mAP and Rank1 on the first seen domain Market-1501. The training order is Market$\to$Cuhk-Sysu$\to$MSMT17.
• Figure 6: Ablation study on non-forgetting evaluation with mAP and Rank1 on the first seen domain Market-1501.