Multi-Person Pose Estimation Evaluation Using Optimal Transportation and Improved Pose Matching

TL;DR

This paper proposes Optimal Correction Cost for pose (OCpose), which evaluates detected poses against pose annotations as an optimal transportation and provides a different perspective assessment than other confidence ranking-based metrics.

Abstract

In Multi-Person Pose Estimation, many metrics place importance on ranking of pose detection confidence scores. Current metrics tend to disregard false-positive poses with low confidence, focusing primarily on a larger number of high-confidence poses. Consequently, these metrics may yield high scores even when many false-positive poses with low confidence are detected. For fair evaluation taking into account a tradeoff between true-positive and false-positive poses, this paper proposes Optimal Correction Cost for pose (OCpose), which evaluates detected poses against pose annotations as an optimal transportation. For the fair tradeoff between true-positive and false-positive poses, OCpose equally evaluates all the detected poses regardless of their confidence scores. In OCpose, on the other hand, the confidence score of each pose is utilized to improve the reliability of matching scores between the estimated pose and pose annotations. As a result, OCpose provides a different perspective assessment than other confidence ranking-based metrics.

Figures (7)

• Figure 1: Unfair evaluation in mAP, which is a major evaluation metric using the confidence rank. Lower thresholds lead to high scores (AP) while producing many false-positives (FP).
• Figure 2: Difference between mAP and OCpose. Red circles highlight false positives. mAP favors Estimator A despite many false positives, whereas OCpose prioritizes Estimator B with fewer false detections.
• Figure 3: Overview of OCpose. Given GT poses, masks, crowd masks, and estimated poses, OT costs between them are calculated to determine the evaluation score.
• Figure 4: Comparison of bbox and mask annotations, which are indicated by green rectangles in (b) and green pixels in (c), respectively. While the bbox incorrectly accepts the false pose colored blue in (b), the mask can recognize it as a false positive in (c).
• Figure 5: Confidence-based pose matching. Although several keypoints are located outside the mask, their low confidence reduces the influence on the OKS. In contrast, conventional OKS may penalize such poses regardless of confidence.