Unity is Strength: Unifying Convolutional and Transformeral Features for Better Person Re-Identification

TL;DR

This work addresses image-based person Re-ID by unifying local CNN features with global Transformer representations. It introduces FusionReID, which combines a Dual-branch Feature Extraction (DFE) module with a Dual-attention Mutual Fusion (DMF) mechanism that stacks Local Refinement Units (LRU) and Heterogenous Transmission Modules (HTM) consisting of Shared Encoding Units (SEU) and Mutual Fusion Units (MFU) to enable deep, cross-branch feature fusion. The method is supervised through six outputs using label-smoothed cross-entropy and triplet losses, yielding strong performance on Market1501, DukeMTMC, and MSMT17, with ablations confirming the contributions of each component and the complementarity of CNNs and Transformers. The approach offers a generalizable and effective pathway for combining local and global cues in Re-ID, with practical impact for robust multi-camera surveillance and intelligent transportation systems. Code availability further supports adoption and reproducibility in the field.

Abstract

Person Re-identification (ReID) aims to retrieve the specific person across non-overlapping cameras, which greatly helps intelligent transportation systems. As we all know, Convolutional Neural Networks (CNNs) and Transformers have the unique strengths to extract local and global features, respectively. Considering this fact, we focus on the mutual fusion between them to learn more comprehensive representations for persons. In particular, we utilize the complementary integration of deep features from different model structures. We propose a novel fusion framework called FusionReID to unify the strengths of CNNs and Transformers for image-based person ReID. More specifically, we first deploy a Dual-branch Feature Extraction (DFE) to extract features through CNNs and Transformers from a single image. Moreover, we design a novel Dual-attention Mutual Fusion (DMF) to achieve sufficient feature fusions. The DMF comprises Local Refinement Units (LRU) and Heterogenous Transmission Modules (HTM). LRU utilizes depth-separable convolutions to align deep features in channel dimensions and spatial sizes. HTM consists of a Shared Encoding Unit (SEU) and two Mutual Fusion Units (MFU). Through the continuous stacking of HTM, deep features after LRU are repeatedly utilized to generate more discriminative features. Extensive experiments on three public ReID benchmarks demonstrate that our method can attain superior performances than most state-of-the-arts. The source code is available at https://github.com/924973292/FusionReID.

Figures (9)

• Figure 1: Different unities of CNNs and Transformers. (a) Transformer is embedded into CNN; (b) CNN is embedded into Transformer; (c) Parallel interaction between CNNs and Transformers; (d) Our framework focuses on the deep feature fusion.
• Figure 2: Overall architecture of the proposed FusionReID. With CNN and Transformer backbones, the Dual-branch Feature Extraction (DFE) is used to extract two types of deep features from the same image. Afterwards, the Dual-attention Mutual Fusion (DMF) is adopted to fuse deep features. More specifically, Local Refinement Units (LRU) are utilized to adjust the channel dimensions and spatial sizes. Heterogenous Transmission Modules (HTM) are stacked to enhance the heterogenous features. The framework allows the combination of different backbones, resulting in highly discriminative features.
• Figure 3: Illustrations of our key modules. (a) Details of continuously stacked HTM; (b) SEU for deep feature enhancement; (c) MFU for deep feature fusion. We take the Transformer branch as an example. The CNN branch has a similar structure.
• Figure 4: Various structures in HTM. (a) SEUs in front and MFUs in back; (b) Only shared SEUs are continuously stacked; (c) MFUs in front and SEU in back; (d) Only unshared MFUs are stacked.
• Figure 5: Effects of stacked HTM on MSMT17.