VISUALCENT: Visual Human Analysis using Dynamic Centroid Representation

TL;DR

VisualCent introduces a unified, detector-free framework for multi-person pose estimation and instance segmentation that relies on centroid-based representations. It employs KeyCentroid within a Disk Representation heatmap to locate keypoints and MaskCentroid as a dynamic clustering anchor to assign pixels to person instances, enabling scalable and real-time performance in crowded scenarios. The method is trained end-to-end with KHDR and a dynamic clustering loss, and achieves state-of-the-art or competitive results on COCO and OCHuman for both keypoint and segmentation tasks while reducing computational overhead. The combination of KHDR, KeyCentroid, and DM_c facilitates robust performance under rapid movement and occlusion, making VisualCent suitable for real-time human-centric visual analysis in interactive applications.

Abstract

We introduce VISUALCENT, a unified human pose and instance segmentation framework to address generalizability and scalability limitations to multi person visual human analysis. VISUALCENT leverages centroid based bottom up keypoint detection paradigm and uses Keypoint Heatmap incorporating Disk Representation and KeyCentroid to identify the optimal keypoint coordinates. For the unified segmentation task, an explicit keypoint is defined as a dynamic centroid called MaskCentroid to swiftly cluster pixels to specific human instance during rapid changes in human body movement or significantly occluded environment. Experimental results on COCO and OCHuman datasets demonstrate VISUALCENTs accuracy and real time performance advantages, outperforming existing methods in mAP scores and execution frame rate per second. The implementation is available on the project page.

Figures (6)

• Figure 1: Visual performance of VisualCent. (a) represents KeyCentroid to identify precise keypoint coordinates. (b) represents MaskCentroid with dynamic high confident keypoint to cluster the mask pixels to the correct instance. (c) represents an illustration of a binary mask generated with the help of MaskCentroid (d) exemplifies the instance-level segmentation. (e) shows a unified representation of human pose and estimation (zoom in).
• Figure 2: The overview of the proposed system. $F^*$ indicates the KeyCentroid $K_c$ operation. $F'$ indicates the MaskCentroid $M_c$ operation using the Explicit keypoint $E_k$ predicted by $K_c$.
• Figure 3: The process of obtaining KeyCentroid involves analyzing the keypoint heatmap generated using features extracted from the backbone network.
• Figure 4: The segmentation map is created using MaskCentroid, a dynamic keypoint that clusters mask pixels into instances with high confidence. This approach leverages segmentation features from the backbone network, ensuring precise and spatially coherent pixel assignments.
• Figure 5: Computational cost with the representative sister models. Models are tested on a single Titan RTX.