Argus: Multi-View Egocentric Human Mesh Reconstruction Based on Stripped-Down Wearable mmWave Add-on

TL;DR

Argus, a wearable add-on system based on stripped-down mmWave radars, is the first to achieve egocentric human mesh reconstruction in a multi-view manner and achieves performance comparable to traditional solutions based on high-capability mmWave radars.

Abstract

In this paper, we propose Argus, a wearable add-on system based on stripped-down (i.e., compact, lightweight, low-power, limited-capability) mmWave radars. It is the first to achieve egocentric human mesh reconstruction in a multi-view manner. Compared with conventional frontal-view mmWave sensing solutions, it addresses several pain points, such as restricted sensing range, occlusion, and the multipath effect caused by surroundings. To overcome the limited capabilities of the stripped-down mmWave radars (with only one transmit antenna and three receive antennas), we tackle three main challenges and propose a holistic solution, including tailored hardware design, sophisticated signal processing, and a deep neural network optimized for high-dimensional complex point clouds. Extensive evaluation shows that Argus achieves performance comparable to traditional solutions based on high-capability mmWave radars, with an average vertex error of 6.5 cm, solely using stripped-down radars deployed in a multi-view configuration. It presents robustness and practicality across conditions, such as with unseen users and different host devices.

Figures (25)

• Figure 1: Argus is the first multi-view, egocentric mmWave sensing system enabling continuous HMR, breaking through the limitations of frontal-view solutions.
• Figure 2: System overview. By transferring the knowledge from RGB images as pseudo-labels, a well-trained model will be deployed in the testing phase to perform continuous 3D human mesh reconstruction.
• Figure 3: Block diagram of Argus hardware.
• Figure 4: Argus 3D.
• Figure 5: 3D sensing model.