Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Informative Attention Weights for Person Re-Identification

Yancheng Wang, Nebojsa Jojic, Yingzhen Yang

TL;DR

This work addresses the challenge that existing attention modules in person Re-ID may attend to non-informative image regions. It introduces Reduction of Information Bottleneck (RIB), a distribution-free variational upper bound IBB that can be optimized with SGD to encourage attention weights that correlate more with identity and less with nuisance inputs. RIB is instantiated via Differentiable Channel Selection Attention (DCS-Attention) for self-attention and extended to existing channel attentions (RIB-CA), and is applied through fixed-backbone and NAS-backed backbones (RIB-DCS-FB, RIB-DCS-DNAS, RIB-CA-FB, RIB-CA-DNAS). Across Market-1501, DukeMTMC, MSMT17, and CUHK03, including occluded and cross-domain scenarios, RIB variants consistently outperform baselines, with efficient training and NAS-assisted backbone search contributing to strong, scalable performance improvements.

Abstract

Attention mechanisms have been widely used in deep learning, and recent efforts have been devoted to incorporating attention modules into deep neural networks (DNNs) for person Re-Identification (Re-ID) to enhance their discriminative feature learning capabilities. Existing attention modules, including self-attention and channel attention, learn attention weights that quantify the importance of feature tokens or feature channels. However, existing attention methods do not explicitly ensure that the attention weights are informative for predicting the identity of the person in the input image, and may consequently introduce noisy information from the input image. To address this issue, we propose a novel method termed Reduction of Information Bottleneck loss (RIB), motivated by the principle of the Information Bottleneck (IB). A novel distribution-free and efficient variational upper bound for the IB loss (IBB), which can be optimized by standard SGD, is derived and incorporated into the training loss of the RIB models. RIB is applied to DNNs with self-attention modules through a novel Differentiable Channel Selection Attention module, or DCS-Attention, that selects the most informative channels for computing attention weights, leading to competitive models termed RIB-DCS. RIB is also incorporated into DNNs with existing channel attention modules to promote the learning of informative channel attention weights, leading to models termed RIB-CA. Both RIB-DCS and RIB-CA are applied to fixed neural network backbones and learnable backbones with Differentiable Neural Architecture Search (DNAS). Extensive experiments on multiple person Re-ID benchmarks show that RIB significantly enhances the prediction accuracy of DNNs for person Re-ID, even for the occluded person Re-ID.

Learning Informative Attention Weights for Person Re-Identification

TL;DR

This work addresses the challenge that existing attention modules in person Re-ID may attend to non-informative image regions. It introduces Reduction of Information Bottleneck (RIB), a distribution-free variational upper bound IBB that can be optimized with SGD to encourage attention weights that correlate more with identity and less with nuisance inputs. RIB is instantiated via Differentiable Channel Selection Attention (DCS-Attention) for self-attention and extended to existing channel attentions (RIB-CA), and is applied through fixed-backbone and NAS-backed backbones (RIB-DCS-FB, RIB-DCS-DNAS, RIB-CA-FB, RIB-CA-DNAS). Across Market-1501, DukeMTMC, MSMT17, and CUHK03, including occluded and cross-domain scenarios, RIB variants consistently outperform baselines, with efficient training and NAS-assisted backbone search contributing to strong, scalable performance improvements.

Abstract

Attention mechanisms have been widely used in deep learning, and recent efforts have been devoted to incorporating attention modules into deep neural networks (DNNs) for person Re-Identification (Re-ID) to enhance their discriminative feature learning capabilities. Existing attention modules, including self-attention and channel attention, learn attention weights that quantify the importance of feature tokens or feature channels. However, existing attention methods do not explicitly ensure that the attention weights are informative for predicting the identity of the person in the input image, and may consequently introduce noisy information from the input image. To address this issue, we propose a novel method termed Reduction of Information Bottleneck loss (RIB), motivated by the principle of the Information Bottleneck (IB). A novel distribution-free and efficient variational upper bound for the IB loss (IBB), which can be optimized by standard SGD, is derived and incorporated into the training loss of the RIB models. RIB is applied to DNNs with self-attention modules through a novel Differentiable Channel Selection Attention module, or DCS-Attention, that selects the most informative channels for computing attention weights, leading to competitive models termed RIB-DCS. RIB is also incorporated into DNNs with existing channel attention modules to promote the learning of informative channel attention weights, leading to models termed RIB-CA. Both RIB-DCS and RIB-CA are applied to fixed neural network backbones and learnable backbones with Differentiable Neural Architecture Search (DNAS). Extensive experiments on multiple person Re-ID benchmarks show that RIB significantly enhances the prediction accuracy of DNNs for person Re-ID, even for the occluded person Re-ID.
Paper Structure (36 sections, 3 theorems, 3 equations, 11 figures, 17 tables, 3 algorithms)

This paper contains 36 sections, 3 theorems, 3 equations, 11 figures, 17 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Person Re-identification and Attention Modules
  4. Neural Architecture Search
  5. Related Works about Information Bottleneck (IB)
  6. Proposed Approach
  7. RIB: Reduction of Information Bottleneck Loss
  8. Application of RIB to Self-Attention and Channel Attention
  9. RIB with Fixed Backbone and Learnable Backbone
  10. Experiments
  11. Datasets and Evaluation Metrics
  12. Implementation Details
  13. RIB-DCS for Person Re-ID
  14. RIB-CA for Person Re-ID
  15. Ablation Study on the Effects of IBB
  16. ...and 21 more sections

Key Result

Theorem 3.1

Let $\textup{Pr}\left[X \in y\right] = \sum_{i=1}^n {\rm 1}{\rm I}_{\left\{y_i = y\right\}}/n \coloneqq p_y$ be the prior probability for every $y \in [C]$, we have IB( W) IBB( W) , where 1n ∑_i=1^n ∑_a=1^C ∑_y=1^C 1 I_{y_i = y} ϕ(F_i, a) ( 1 I_{y_i = y}p_y Q(F ∈ a| Y=y) ).

Figures (11)

  • Figure 1: Figures (a)-(b) illustrate the Grad-CAM visualization for two images from the occluded person Re-ID dataset, Occluded-Duke miao2019pose, for an attention-based model, SPT tan2024occluded, and RIB-DCS-FB (SPT). Figures (c)-(d) illustrate the heatmaps of the attention weights corresponding to a query token computed from the first transformer block in SPT and RIB-DCS-FB (SPT). The query token for both examples is selected from the boundary of the arm of the person in the images, which is critical for identifying a person.
  • Figure 2: Figures (a)-(b) illustrate the Grad-CAM heatmaps for TransReID, TransReID+DCS-Attention, and RIB-DCS-FB (TransReID). TransReID+DCS-Attention replaces the self-attention modules in TransRe-ID with the DCS-Attention modules. Figure (c) illustrates the Grad-CAM heatmaps for TransReID, TransReID+SE, and RIB-CA-FB (TransReID+SE). The SE attention modules hu2018squeeze are inserted after each transformer block in TransReID+SE. More visualization results are deferred to Section \ref{['sec:grad_cam']} of the supplementary.
  • Figure 3: Figure (a) illustrates the framework of the DCS-Attention module. Figure (b) illustrates the pipeline for the RIB-DCS model. The DCS-Attention modules are inserted after each residual block. In a RIB-CA model, the DCS-Attention modules are replaced with existing channel attention modules, such as SE hu2018squeeze, CBAM CBAM, and MCA MCA. IBB is added as a regularization term in addition to the triplet loss and the cross-entropy (CE) loss.
  • Figure 4: Grad-CAM visualization results for TransReID, TransReID+DCS-Attention, and RIB-DCS-FB (TransReID). TransReID+DCS-Attention replaces all the self-attention modules in TransReID with the DCS-Attention modules.
  • Figure 5: Grad-CAM visualization results for MobileNetV2, MobileNetV2+DCS-Attention, and RIB-DCS-FB (MobileNetV2). MobileNetV2+DCS-Attention inserts the DCS-Attention module after each of its residual blocks.
  • ...and 6 more figures

Theorems & Definitions (6)

  • Theorem 3.1
  • Lemma F.1
  • proof
  • Lemma F.2
  • proof
  • proof : Proof of Theorem 3.1