Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dite-HRNet: Dynamic Lightweight High-Resolution Network for Human Pose Estimation

Qun Li, Ziyi Zhang, Fu Xiao, Feng Zhang, Bir Bhanu

TL;DR

Dite-HRNet tackles the inefficiency of high-resolution pose estimators by introducing two dynamic lightweight blocks that operate within a HRNet-like backbone: Dynamic Split Convolution (DSC) and Adaptive Context Modeling (ACM). DSC combines a Split-Concat-Shuffle module with Dynamic Kernel Aggregation to produce input-dependent, multi-scale convolutions, while ACM (via Dense Context Modeling and Global Context Modeling) injects both cross-resolution and per-resolution contextual information. The approach yields Dynamic Multi-scale Context (DMC) and Dynamic Global Context (DGC) blocks that leverage parallel multi-resolution branches to preserve high resolution while reducing cost. Extensive experiments on COCO and MPII demonstrate superior efficiency and competitive or superior accuracy compared with state-of-the-art lightweight networks and larger baselines, highlighting the method's practical impact for real-time or on-device pose estimation and its potential to extend to other multi-scale architectures.

Abstract

A high-resolution network exhibits remarkable capability in extracting multi-scale features for human pose estimation, but fails to capture long-range interactions between joints and has high computational complexity. To address these problems, we present a Dynamic lightweight High-Resolution Network (Dite-HRNet), which can efficiently extract multi-scale contextual information and model long-range spatial dependency for human pose estimation. Specifically, we propose two methods, dynamic split convolution and adaptive context modeling, and embed them into two novel lightweight blocks, which are named dynamic multi-scale context block and dynamic global context block. These two blocks, as the basic component units of our Dite-HRNet, are specially designed for the high-resolution networks to make full use of the parallel multi-resolution architecture. Experimental results show that the proposed network achieves superior performance on both COCO and MPII human pose estimation datasets, surpassing the state-of-the-art lightweight networks. Code is available at: https://github.com/ZiyiZhang27/Dite-HRNet.

Dite-HRNet: Dynamic Lightweight High-Resolution Network for Human Pose Estimation

TL;DR

Dite-HRNet tackles the inefficiency of high-resolution pose estimators by introducing two dynamic lightweight blocks that operate within a HRNet-like backbone: Dynamic Split Convolution (DSC) and Adaptive Context Modeling (ACM). DSC combines a Split-Concat-Shuffle module with Dynamic Kernel Aggregation to produce input-dependent, multi-scale convolutions, while ACM (via Dense Context Modeling and Global Context Modeling) injects both cross-resolution and per-resolution contextual information. The approach yields Dynamic Multi-scale Context (DMC) and Dynamic Global Context (DGC) blocks that leverage parallel multi-resolution branches to preserve high resolution while reducing cost. Extensive experiments on COCO and MPII demonstrate superior efficiency and competitive or superior accuracy compared with state-of-the-art lightweight networks and larger baselines, highlighting the method's practical impact for real-time or on-device pose estimation and its potential to extend to other multi-scale architectures.

Abstract

A high-resolution network exhibits remarkable capability in extracting multi-scale features for human pose estimation, but fails to capture long-range interactions between joints and has high computational complexity. To address these problems, we present a Dynamic lightweight High-Resolution Network (Dite-HRNet), which can efficiently extract multi-scale contextual information and model long-range spatial dependency for human pose estimation. Specifically, we propose two methods, dynamic split convolution and adaptive context modeling, and embed them into two novel lightweight blocks, which are named dynamic multi-scale context block and dynamic global context block. These two blocks, as the basic component units of our Dite-HRNet, are specially designed for the high-resolution networks to make full use of the parallel multi-resolution architecture. Experimental results show that the proposed network achieves superior performance on both COCO and MPII human pose estimation datasets, surpassing the state-of-the-art lightweight networks. Code is available at: https://github.com/ZiyiZhang27/Dite-HRNet.
Paper Structure (29 sections, 6 equations, 3 figures, 6 tables)

This paper contains 29 sections, 6 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Lightweight Human Pose Estimation.
  4. Efficient CNN Blocks.
  5. Spatial Dependency Modeling.
  6. Dynamic CNN Architectures.
  7. Dite-HRNet
  8. Block Design.
  9. Configuration.
  10. Dynamic Split Convolution (DSC)
  11. Split-Concat-Shuffle (SCS) Module.
  12. Dynamic Kernel Aggregation (DKA).
  13. Complementation.
  14. Adaptive Context Modeling (ACM)
  15. Dense Context Modeling (DCM).
  16. ...and 14 more sections

Figures (3)

  • Figure 1: Overall architecture of Dite-HRNet. It uses HRNet sun:hrnet as backbone, adapted by two proposed dynamic blocks. The cross-resolution modules are marked with blue areas.
  • Figure 2: Structures of Dynamic Multi-scale Context (DMC) block and Dynamic Global Context (DGC) block. Dynamic Kernel Aggregation (DKA) is applied to Dynamic Split Convolution (DSC) in the DMC block and each convolution in the DGC block. DCM (Dense Context Modeling) and GCM (Global Context Modeling) are two instantiations of our proposed Adaptive Context Modeling (ACM).
  • Figure 3: Procedure of adaptive context pooling. $C$ denotes the number of channels. $H$ and $W$ denote the current height and width of features. $\widehat{H}$ and $\widehat{W}$ denote the target height and width of features. $\otimes$ denotes matrix multiplication.