Kinematic Motion Retargeting for Contact-Rich Anthropomorphic Manipulations

TL;DR

The paper tackles retargeting dense, contact-rich hand-object motions across morphologically diverse hands by formulating the transfer as a non-isometric shape matching problem using dense contact correspondences. It introduces an atlas-based framework built on local charts with logmap/expmap representations, augmented by virtual markers and axial curves, to transfer contact distributions and synthesize target-hand trajectories through a three-stage optimization (initial estimation, refinement, spline fitting) solved with MMA. The approach is validated on 30 demonstrations across five hands and six motions, showing robust performance and favorable comparisons to contact-free baselines, along with practical extensions for design visualization and object substitution. The method emphasizes kinematics over dynamics, yielding reliable retargeting with interpretable artist-driven annotations, while acknowledging limitations such as atlas discontinuities and challenges with highly divergent morphologies. Overall, the work provides a standardized, extensible pipeline that enables rapid rigging, design evaluation, and potential downstream benefits for robotics and animation through improved contact-aware motion transfer.

Abstract

Hand motion capture data is now relatively easy to obtain, even for complicated grasps; however this data is of limited use without the ability to retarget it onto the hands of a specific character or robot. The target hand may differ dramatically in geometry, number of degrees of freedom (DOFs), or number of fingers. We present a simple, but effective framework capable of kinematically retargeting multiple human hand-object manipulations from a publicly available dataset to a wide assortment of kinematically and morphologically diverse target hands through the exploitation of contact areas. We do so by formulating the retarget operation as a non-isometric shape matching problem and use a combination of both surface contact and marker data to progressively estimate, refine, and fit the final target hand trajectory using inverse kinematics (IK). Foundational to our framework is the introduction of a novel shape matching process, which we show enables predictable and robust transfer of contact data over full manipulations while providing an intuitive means for artists to specify correspondences with relatively few inputs. We validate our framework through thirty demonstrations across five different hand shapes and six motions of different objects. We additionally compare our method against existing hand retargeting approaches. Finally, we demonstrate our method enabling novel capabilities such as object substitution and the ability to visualize the impact of design choices over full trajectories.

Figures (19)

• Figure 1: All hands used in our experiments, including the (a) source MANO hand, (b) an alternate human hand, (c) a witch hand, (d) an alien hand, (e) a custom prosthetic hand, and (f) the Allegro Hand. (g) We retarget demonstrations performed by the source hand to all these hands by procedurally transferring contact areas over the entire time series via shape matching.
• Figure 2: Overview of our retargeting framework. (a) Our approach requires inputs of accurate meshes of the original object and source hand, per-frame contacts on either the object or source hand, and a complete motion sequence of the source hand. (b) To perform the retarget, we require a skeleton-driven target hand mesh as well as a set of artist-annotated corresponding virtual markers and axial curves. (c) After recovering a dense set of contacts between the object and source hand, we transfer contacts across the entire time series and (d) use the virtual markers and transferred contacts to synthesize motion for the target hand from scratch.
• Figure 3: Virtual markers can be configured as traditional single-point one-to-one, or alternatively (a) heterogeneous many-to-one, or (b) dense, area-based configurations. Configuration (a) can be utilized to model uncertainty between virtual marker locations between differing source and target hands, which can be useful when finger link lengths are different sizes. Configuration (b) can be used to weight the importance of matching the deformed hand states over large regions, which can be useful when deformation behaviors diverge despite similar link lengths.
• Figure 4: Single point virtual marker configuration on the source MANO hand and area based corresponding marker sets on all other hands. We use a manifold wrapper of the Allegro Hand for contact processing.
• Figure 5: (Top Row) Object contacts, (Second Row) source hand contacts, (Third Row) computed target hand contacts, and (Bottom Row) source and retargeted hand motion for four different stages of a complex phone manipulation: (First Column) table pickup, (Second Column) in-hand dialing, (Third Column) holding for use, and (Last Column) movement back towards the table for release. Although poses and contact distributions vary dramatically during the manipulation, our method can successfully produce target hand motion by using source hand contact distributions as a foundational retargeting medium.