POSE: Pose estimation Of virtual Sync Exhibit system

TL;DR

A platform that can control virtual avatars through pose estimation to identify the movements of real people, and a multi-process to achieve modularization and reduce the overall latency is developed.

Abstract

This work is a portable MetaVerse implementation, and we use 3D pose estimation with AI to make virtual avatars do synchronized actions and interact with the environment. The motivation is that we find it inconvenient to use joysticks and sensors when playing with fitness rings. In order to replace joysticks and reduce costs, we developed a platform that can control virtual avatars through pose estimation to identify the movements of real people, and we also implemented a multi-process to achieve modularization and reduce the overall latency.

Figures (11)

• Figure 1: Project architecture diagram
• Figure 2: Human Pose Estimation
• Figure 3: The constraints and the avatar operation of Inverse Kinematics
• Figure 4: The converting process of joint vectors
• Figure 5: Results of pose estimation model