Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniHands: Unifying Various Wild-Collected Keypoints for Personalized Hand Reconstruction

Menghe Zhang, Joonyeoup Kim, Yangwen Liang, Shuangquan Wang, Kee-Bong Song

TL;DR

UniHands is a novel method for creating standardized yet personalized hand models from wild-collected keypoints from diverse sources, using the widely-adopted parametric models MANO and NIMBLE, providing a more scalable and versatile solution.

Abstract

Accurate hand motion capture and standardized 3D representation are essential for various hand-related tasks. Collecting keypoints-only data, while efficient and cost-effective, results in low-fidelity representations and lacks surface information. Furthermore, data inconsistencies across sources challenge their integration and use. We present UniHands, a novel method for creating standardized yet personalized hand models from wild-collected keypoints from diverse sources. Unlike existing neural implicit representation methods, UniHands uses the widely-adopted parametric models MANO and NIMBLE, providing a more scalable and versatile solution. It also derives unified hand joints from the meshes, which facilitates seamless integration into various hand-related tasks. Experiments on the FreiHAND and InterHand2.6M datasets demonstrate its ability to precisely reconstruct hand mesh vertices and keypoints, effectively capturing high-degree articulation motions. Empirical studies involving nine participants show a clear preference for our unified joints over existing configurations for accuracy and naturalism (p-value 0.016).

UniHands: Unifying Various Wild-Collected Keypoints for Personalized Hand Reconstruction

TL;DR

UniHands is a novel method for creating standardized yet personalized hand models from wild-collected keypoints from diverse sources, using the widely-adopted parametric models MANO and NIMBLE, providing a more scalable and versatile solution.

Abstract

Accurate hand motion capture and standardized 3D representation are essential for various hand-related tasks. Collecting keypoints-only data, while efficient and cost-effective, results in low-fidelity representations and lacks surface information. Furthermore, data inconsistencies across sources challenge their integration and use. We present UniHands, a novel method for creating standardized yet personalized hand models from wild-collected keypoints from diverse sources. Unlike existing neural implicit representation methods, UniHands uses the widely-adopted parametric models MANO and NIMBLE, providing a more scalable and versatile solution. It also derives unified hand joints from the meshes, which facilitates seamless integration into various hand-related tasks. Experiments on the FreiHAND and InterHand2.6M datasets demonstrate its ability to precisely reconstruct hand mesh vertices and keypoints, effectively capturing high-degree articulation motions. Empirical studies involving nine participants show a clear preference for our unified joints over existing configurations for accuracy and naturalism (p-value 0.016).

Paper Structure

This paper contains 8 sections, 3 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Mesh Reconstruction
  4. Unified Joints from Hand Mesh
  5. Experiments
  6. Evaluation of Mesh Reconstruction
  7. Study of Joints from Mesh
  8. Discussion and Conclusion

Figures (2)

  • Figure 1: Method illustrations. (a) Our unified joints (middle) combines the strengths of both MANO (left) and NIMBLE (right) keypoints. (b) Our MLP derives NIMBLE joints from MANO mesh.
  • Figure 2: Evaluation Results. (a) Evaluations of three tasks across three joint sets, showing raw values (left) and mapped ranks (right). (b) Results from the modified System Usability Scale (SUS)/Task Load Evaluation (TLE) are weighted and displayed on a spectrum: lower ratings appear on the left in warmer hues, while higher ratings are on the right in cooler hues.