PaMM: Pose-aware Multi-shot Matching for Improving Person Re-identification

TL;DR

This work tackles cross-view person re-identification under pose and viewpoint variability by introducing PaMM, a Pose-aware Multi-shot Matching framework. PaMM first estimates camera viewpoints and person poses from auto-calibration cues, then builds four-pose multi-pose models (front, right, back, left) filtered by a sample confidence score, and finally performs pose-weighted multi-shot matching with learned weights. The matching uses a cross-pose distance $x_{p_i q_j}^{k,l}$ and aggregates it via a learned weight vector $\mathbf{w}$ into $C(\mathcal{M}^k, \mathcal{M}^l)$, with weights trained by a binary SVM on data from CUHK02 and VIPeR. Experiments on 3DPeS, PRID, and iLIDS show that PaMM consistently improves over strong baselines, demonstrating the value of incorporating pose priors and multi-shot, pose-aware representations for robust re-identification in diverse viewpoints. The framework is flexible and can integrate with various feature descriptors and metric learners, and the authors provide public code to support reproducibility.

Abstract

Person re-identification is the problem of recognizing people across different images or videos with non-overlapping views. Although there has been much progress in person re-identification over the last decade, it remains a challenging task because appearances of people can seem extremely different across diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person re-identification by analyzing camera viewpoints and person poses in a so-called Pose-aware Multi-shot Matching (PaMM), which robustly estimates people's poses and efficiently conducts multi-shot matching based on pose information. Experimental results using public person re-identification datasets show that the proposed methods outperform state-of-the-art methods and are promising for person re-identification from diverse viewpoints and pose variances.

Figures (12)

• Figure 1: Challenging in person re-identification due to person appearance changes. Person appearance changes depending on the camera viewpoint and the pose of a person. Pairs of red squares have same locations but show different appearances due to pose variations of people.
• Figure 2: The proposed pose-aware multi-shot matching (PaMM) framework for person re-identification. First, the pose of each person is estimated. Second, a multi-pose model is generated. Finally two multi-pose models are matched based on pre-trained matching weights. The thicknesses of lines indicate the matching weights.
• Figure 3: Person pose estimation: (left) estimated 3D structure and person poses along the path, (right) corresponding 2D images and appearances grouped by poses.
• Figure 4: (left) initial pose angle, (middle) smoothing result in Cartesian coordinates, (right) smoothing result in polar coordinates.
• Figure 5: Sample confidence under various conditions (best viewed in color and at a high resolution).