Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Acoustic-based 3D Human Pose Estimation Robust to Human Position

Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa

TL;DR

This paper proposes a novel method composed of a position discriminator and reverberation-resistant model that utilizes acoustic signals before the estimation target time as references to enhance robustness against the variations in sound arrival times due to diffraction and reflection.

Abstract

This paper explores the problem of 3D human pose estimation from only low-level acoustic signals. The existing active acoustic sensing-based approach for 3D human pose estimation implicitly assumes that the target user is positioned along a line between loudspeakers and a microphone. Because reflection and diffraction of sound by the human body cause subtle acoustic signal changes compared to sound obstruction, the existing model degrades its accuracy significantly when subjects deviate from this line, limiting its practicality in real-world scenarios. To overcome this limitation, we propose a novel method composed of a position discriminator and reverberation-resistant model. The former predicts the standing positions of subjects and applies adversarial learning to extract subject position-invariant features. The latter utilizes acoustic signals before the estimation target time as references to enhance robustness against the variations in sound arrival times due to diffraction and reflection. We construct an acoustic pose estimation dataset that covers diverse human locations and demonstrate through experiments that our proposed method outperforms existing approaches.

Acoustic-based 3D Human Pose Estimation Robust to Human Position

TL;DR

This paper proposes a novel method composed of a position discriminator and reverberation-resistant model that utilizes acoustic signals before the estimation target time as references to enhance robustness against the variations in sound arrival times due to diffraction and reflection.

Abstract

This paper explores the problem of 3D human pose estimation from only low-level acoustic signals. The existing active acoustic sensing-based approach for 3D human pose estimation implicitly assumes that the target user is positioned along a line between loudspeakers and a microphone. Because reflection and diffraction of sound by the human body cause subtle acoustic signal changes compared to sound obstruction, the existing model degrades its accuracy significantly when subjects deviate from this line, limiting its practicality in real-world scenarios. To overcome this limitation, we propose a novel method composed of a position discriminator and reverberation-resistant model. The former predicts the standing positions of subjects and applies adversarial learning to extract subject position-invariant features. The latter utilizes acoustic signals before the estimation target time as references to enhance robustness against the variations in sound arrival times due to diffraction and reflection. We construct an acoustic pose estimation dataset that covers diverse human locations and demonstrate through experiments that our proposed method outperforms existing approaches.

Paper Structure

This paper contains 18 sections, 4 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Pose Estimation Module
  5. Position Discriminator Module
  6. Data Augmentation
  7. Experimental Settings
  8. Dataset and Setup
  9. Baselines
  10. Evaluation Metrics
  11. Implementation Details
  12. Experimental Results
  13. Comparison with Other Baselines
  14. Ablative Analysis
  15. Comparison by Input Size
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Left: acoustic-signal based human pose estimation with different target subject positions. Unlike the existing method that (a) utilizes obscured acoustic signals by the target subject positioned along the line between the microphone and loudspeaker, we aim to estimate (b) poses with the subject who is away from this line, which is the challenging task due to the presence of signal reflection and diffraction. Right: principal component analysis of acoustic features depending on a subject's standing position.
  • Figure 2: Proposed Framework.
  • Figure 3: Experimental setup and equipment
  • Figure 4: The qualitative results of 75 cm and 100 cm from the line
  • Figure 5: Comparison by Plain Clothes Dataset