Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LGM-Pose: A Lightweight Global Modeling Network for Real-time Human Pose Estimation

Biao Guo, Cong Zhou, Fangmin Guo, Xiaonan Luo, Guibo Luo, Feng Zhang

TL;DR

LGM-Pose introduces a near single-branch lightweight network for real-time multi-person 2D pose estimation. It leverages a MobileViM block with a Lightweight Attentional Representation Module and Non-Parametric Transformation Operations to capture global context, complemented by SFusion for efficient multi-scale feature fusion. Across COCO and MPII, it achieves state-of-the-art efficiency among lightweight models, with competitive accuracy and significantly reduced parameters and latency, including favorable CPU performance for edge deployment. The work demonstrates that well-designed global modeling within a single-branch CNN can match or exceed multi-branch architectures while enabling real-time inference on resource-constrained devices.

Abstract

Most of the current top-down multi-person pose estimation lightweight methods are based on multi-branch parallel pure CNN network architecture, which often struggle to capture the global context required for detecting semantically complex keypoints and are hindered by high latency due to their intricate and redundant structures. In this article, an approximate single-branch lightweight global modeling network (LGM-Pose) is proposed to address these challenges. In the network, a lightweight MobileViM Block is designed with a proposed Lightweight Attentional Representation Module (LARM), which integrates information within and between patches using the Non-Parametric Transformation Operation(NPT-Op) to extract global information. Additionally, a novel Shuffle-Integrated Fusion Module (SFusion) is introduced to effectively integrate multi-scale information, mitigating performance degradation often observed in single-branch structures. Experimental evaluations on the COCO and MPII datasets demonstrate that our approach not only reduces the number of parameters compared to existing mainstream lightweight methods but also achieves superior performance and faster processing speeds.

LGM-Pose: A Lightweight Global Modeling Network for Real-time Human Pose Estimation

TL;DR

LGM-Pose introduces a near single-branch lightweight network for real-time multi-person 2D pose estimation. It leverages a MobileViM block with a Lightweight Attentional Representation Module and Non-Parametric Transformation Operations to capture global context, complemented by SFusion for efficient multi-scale feature fusion. Across COCO and MPII, it achieves state-of-the-art efficiency among lightweight models, with competitive accuracy and significantly reduced parameters and latency, including favorable CPU performance for edge deployment. The work demonstrates that well-designed global modeling within a single-branch CNN can match or exceed multi-branch architectures while enabling real-time inference on resource-constrained devices.

Abstract

Most of the current top-down multi-person pose estimation lightweight methods are based on multi-branch parallel pure CNN network architecture, which often struggle to capture the global context required for detecting semantically complex keypoints and are hindered by high latency due to their intricate and redundant structures. In this article, an approximate single-branch lightweight global modeling network (LGM-Pose) is proposed to address these challenges. In the network, a lightweight MobileViM Block is designed with a proposed Lightweight Attentional Representation Module (LARM), which integrates information within and between patches using the Non-Parametric Transformation Operation(NPT-Op) to extract global information. Additionally, a novel Shuffle-Integrated Fusion Module (SFusion) is introduced to effectively integrate multi-scale information, mitigating performance degradation often observed in single-branch structures. Experimental evaluations on the COCO and MPII datasets demonstrate that our approach not only reduces the number of parameters compared to existing mainstream lightweight methods but also achieves superior performance and faster processing speeds.

Paper Structure

This paper contains 23 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Human pose estimation
  4. Lightweight Human Pose Estimation
  5. Method
  6. Architecture
  7. MobileViM
  8. LARM
  9. Non-Parametric Transformation Operation
  10. SFusion
  11. Experiment
  12. Dataset & Evaluation metric
  13. Dataset
  14. Evaluation metric
  15. Implementation Details
  16. ...and 8 more sections

Figures (4)

  • Figure 1: The framework of LGM-Pose. LGM-Pose consists of MobileNetV2 Block, MobileViM Block, SFusion, Deconv, and a predict head. $C_{\text{key}}$ means the number of key points in the corresponding data set; "s" denotes the stride, which controls the sampling interval.
  • Figure 2: LARM structure diagram
  • Figure 3: (a) separable using depth, (b) conventional convolution, and (c) SFusion.
  • Figure 4: Visualization images from the MS-COCO val set, with the first row being the original images, the second row showing the results of Lite-HRNet, the third row displaying the results of Dite-HRNet, and the fourth row presenting the results of LGM-Pose.