A Unified Deep Semantic Expansion Framework for Domain-Generalized Person Re-identification

TL;DR

Upper Deep Semantic Expansion, a novel framework that unifies implicit and explicit semantic feature expansion techniques in a single framework to mitigate this early over-fitting and achieve a new state-of-the-art (SOTA) in all DG-ReID benchmarks is proposed.

Abstract

Supervised Person Re-identification (Person ReID) methods have achieved excellent performance when training and testing within one camera network. However, they usually suffer from considerable performance degradation when applied to different camera systems. In recent years, many Domain Adaptation Person ReID methods have been proposed, achieving impressive performance without requiring labeled data from the target domain. However, these approaches still need the unlabeled data of the target domain during the training process, making them impractical in many real-world scenarios. Our work focuses on the more practical Domain Generalized Person Re-identification (DG-ReID) problem. Given one or more source domains, it aims to learn a generalized model that can be applied to unseen target domains. One promising research direction in DG-ReID is the use of implicit deep semantic feature expansion, and our previous method, Domain Embedding Expansion (DEX), is one such example that achieves powerful results in DG-ReID. However, in this work we show that DEX and other similar implicit deep semantic feature expansion methods, due to limitations in their proposed loss function, fail to reach their full potential on large evaluation benchmarks as they have a tendency to saturate too early. Leveraging on this analysis, we propose Unified Deep Semantic Expansion, our novel framework that unifies implicit and explicit semantic feature expansion techniques in a single framework to mitigate this early over-fitting and achieve a new state-of-the-art (SOTA) in all DG-ReID benchmarks. Further, we apply our method on more general image retrieval tasks, also surpassing the current SOTA in all of these benchmarks by wide margins.

Figures (6)

• Figure 1: DEX, our adaptation of implicit semantic expansion to DG-ReID. DEX improves exploration of the domain space by implicitly projecting training points in the directions of the domain distribution.
• Figure 2: Increasing DEX strength decreases the distance between class weights. Best viewed in color.
• Figure 3: Naively combining explicit and implicit semantic expansion yields poor results as they disrupt each other.
• Figure 4: Our method UDSX unifies implicit and explicit semantic expansion in 3 parts: (1) Data Semantic Decoupling (DSD) isolates Domain Embedding Expansion (DEX) and Progressive Spatio-Temporal Expansion (PSTE) into dedicated streams, (2) PSTE is the engine behind the explicit semantic expansion, and (3) Contrastive-Stream Reunification (CSR) reunifies the streams at the end and more details on this component are illustrated in \ref{['fig:contrastive-stream-reunification-overview']}. Best viewed in color.
• Figure 5: Components of PSTE illustrated. (a) Intermediate features are perturbed based on the domain that they come from, allowing the semantics of each domain to be better explored over time. (b) PTE: We start by randomly selecting among early intermediate features to perturb, and gradually over time expand the selection set to the later layers as the model begins to understand higher-level semantics. (c) ABS: Only selected channels are perturbed, based on their activation values falling within a pre-defined quantile: channels that are either very important or meaningless are left unperturbed. For more details, refer to Section \ref{['subsec:method-semantic-expansion-policy']} and Algorithm \ref{['alg:semantic-expansion-policy']}