Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReConForM : Real-time Contact-aware Motion Retargeting for more Diverse Character Morphologies

Théo Cheynel, Thomas Rossi, Baptiste Bellot-Gurlet, Damien Rohmer, Marie-Paule Cani

TL;DR

ReConForM tackles the challenge of preserving semantic motion semantics when retargeting between characters with different morphologies by introducing a sparse key-vertex mesh embedding and time-varying pose descriptors. An adaptive weighting scheme selects and weights the most relevant constraints over time, enabling real-time optimization over joint rotations and root position to match source motion semantics while minimizing penetration and jerk. The approach demonstrates superior contact accuracy and motion smoothness across diverse characters, with real-time performance and robust extensions to multi-character interactions and non-flat terrains. Practical impact includes enabling diverse character morphologies in film, games, and VR with interactive control and efficient computation. The study also provides a curated evaluation dataset and comprehensive user studies to validate semantic preservation and perceptual quality.

Abstract

Preserving semantics, in particular in terms of contacts, is a key challenge when retargeting motion between characters of different morphologies. Our solution relies on a low-dimensional embedding of the character's mesh, based on rigged key vertices that are automatically transferred from the source to the target. Motion descriptors are extracted from the trajectories of these key vertices, providing an embedding that contains combined semantic information about both shape and pose. A novel, adaptive algorithm is then used to automatically select and weight the most relevant features over time, enabling us to efficiently optimize the target motion until it conforms to these constraints, so as to preserve the semantics of the source motion. Our solution allows extensions to several novel use-cases where morphology and mesh contacts were previously overlooked, such as multi-character retargeting and motion transfer on uneven terrains. As our results show, our method is able to achieve real-time retargeting onto a wide variety of characters. Extensive experiments and comparison with state-of-the-art methods using several relevant metrics demonstrate improved results, both in terms of motion smoothness and contact accuracy.

ReConForM : Real-time Contact-aware Motion Retargeting for more Diverse Character Morphologies

TL;DR

ReConForM tackles the challenge of preserving semantic motion semantics when retargeting between characters with different morphologies by introducing a sparse key-vertex mesh embedding and time-varying pose descriptors. An adaptive weighting scheme selects and weights the most relevant constraints over time, enabling real-time optimization over joint rotations and root position to match source motion semantics while minimizing penetration and jerk. The approach demonstrates superior contact accuracy and motion smoothness across diverse characters, with real-time performance and robust extensions to multi-character interactions and non-flat terrains. Practical impact includes enabling diverse character morphologies in film, games, and VR with interactive control and efficient computation. The study also provides a curated evaluation dataset and comprehensive user studies to validate semantic preservation and perceptual quality.

Abstract

Preserving semantics, in particular in terms of contacts, is a key challenge when retargeting motion between characters of different morphologies. Our solution relies on a low-dimensional embedding of the character's mesh, based on rigged key vertices that are automatically transferred from the source to the target. Motion descriptors are extracted from the trajectories of these key vertices, providing an embedding that contains combined semantic information about both shape and pose. A novel, adaptive algorithm is then used to automatically select and weight the most relevant features over time, enabling us to efficiently optimize the target motion until it conforms to these constraints, so as to preserve the semantics of the source motion. Our solution allows extensions to several novel use-cases where morphology and mesh contacts were previously overlooked, such as multi-character retargeting and motion transfer on uneven terrains. As our results show, our method is able to achieve real-time retargeting onto a wide variety of characters. Extensive experiments and comparison with state-of-the-art methods using several relevant metrics demonstrate improved results, both in terms of motion smoothness and contact accuracy.

Paper Structure

This paper contains 32 sections, 9 equations, 14 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Optimization-based retargeting
  4. Learning-based retargeting
  5. Mesh correspondence
  6. Retargeting datasets
  7. Shape and motion descriptors
  8. Sparse shape encoding and correspondence
  9. Motion encoding via time-varying pose descriptors
  10. Adaptive weighting of constraints
  11. Weighting coefficients
  12. Sparse objective to be optimized
  13. Results
  14. Implementation
  15. Evaluation data
  16. ...and 17 more sections

Figures (14)

  • Figure 1: Examples of issues found in NKN's dataset nkn. Figures \ref{['subfig:BigVegas']} and \ref{['subfig:Kaya']} are called "ground-truth" although they were retargeted using Mixamo, while figures \ref{['subfig:XBot']} and \ref{['subfig:YBot']} show issues without having undergone any retargeting (original source characters for those motions).
  • Figure 2: Chosen location of key-vertices on the template mesh (top-left), and results of the key vertices transfer to several character from Mixamo mixamo. Key vertices are shown with corresponding colored spheres to show the automatic transfer to the different meshes.
  • Figure 3: Top: graphical representation of our method (left: key-vertices transfer presented in Section \ref{['section:shape_encoding']}; middle: motion retargeting process presented in Sections \ref{['section:pose_descriptors']} through \ref{['section:sparse_objective']}). Top-right shows two poses being retargeted onto different characters, with key-vertices shown in red. Bottom shows a complex pose showcasing collisions with the floor as well as self-collisions (the leftmost character is the source character, all the others are target characters from Mixamo).
  • Figure 4: Speed and framerate of our retargeting method
  • Figure 5: User preference across all answers, split by difficulty
  • ...and 9 more figures