Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Switch-JustDance: Benchmarking Whole Body Motion Tracking Policies Using a Commercial Console Game

Jeonghwan Kim, Wontaek Kim, Yidan Lu, Jin Cheng, Fatemeh Zargarbashi, Zicheng Zeng, Zekun Qi, Zhiyang Dou, Nitish Sontakke, Donghoon Baek, Sehoon Ha, Tianyu Li

TL;DR

Switch-JustDance introduces a low-cost, hardware-in-the-loop benchmark that uses the Nintendo Switch Just Dance game to evaluate whole-body humanoid controllers and compare them against human performers. The pipeline reconstructs human motion with GVHMR, retargets to a robot via GMR, and evaluates control policies through the game's Joy-Con-based scoring. The authors validate the Just Dance Score as a reliable, discriminative, and repeatable metric and benchmark three controllers (GMT, TWIST, Any2Track) on hardware and in simulation, revealing a persistent human–robot gap and insights into input-motion effects and sim-to-real transfer. The work provides a practical, reproducible framework for evaluating athletic full-body control in real-world conditions and highlights directions for improving robustness, motion reconstruction, and cross-embodiment benchmarking.

Abstract

Recent advances in whole-body robot control have enabled humanoid and legged robots to perform increasingly agile and coordinated motions. However, standardized benchmarks for evaluating these capabilities in real-world settings, and in direct comparison to humans, remain scarce. Existing evaluations often rely on pre-collected human motion datasets or simulation-based experiments, which limit reproducibility, overlook hardware factors, and hinder fair human-robot comparisons. We present Switch-JustDance, a low-cost and reproducible benchmarking pipeline that leverages motion-sensing console games, Just Dance on the Nintendo Switch, to evaluate robot whole-body control. Using Just Dance on the Nintendo Switch as a representative platform, Switch-JustDance converts in-game choreography into robot-executable motions through streaming, motion reconstruction, and motion retargeting modules and enables users to evaluate controller performance through the game's built-in scoring system. We first validate the evaluation properties of Just Dance, analyzing its reliability, validity, sensitivity, and potential sources of bias. Our results show that the platform provides consistent and interpretable performance measures, making it a suitable tool for benchmarking embodied AI. Building on this foundation, we benchmark three state-of-the-art humanoid whole-body controllers on hardware and provide insights into their relative strengths and limitations.

Switch-JustDance: Benchmarking Whole Body Motion Tracking Policies Using a Commercial Console Game

TL;DR

Switch-JustDance introduces a low-cost, hardware-in-the-loop benchmark that uses the Nintendo Switch Just Dance game to evaluate whole-body humanoid controllers and compare them against human performers. The pipeline reconstructs human motion with GVHMR, retargets to a robot via GMR, and evaluates control policies through the game's Joy-Con-based scoring. The authors validate the Just Dance Score as a reliable, discriminative, and repeatable metric and benchmark three controllers (GMT, TWIST, Any2Track) on hardware and in simulation, revealing a persistent human–robot gap and insights into input-motion effects and sim-to-real transfer. The work provides a practical, reproducible framework for evaluating athletic full-body control in real-world conditions and highlights directions for improving robustness, motion reconstruction, and cross-embodiment benchmarking.

Abstract

Recent advances in whole-body robot control have enabled humanoid and legged robots to perform increasingly agile and coordinated motions. However, standardized benchmarks for evaluating these capabilities in real-world settings, and in direct comparison to humans, remain scarce. Existing evaluations often rely on pre-collected human motion datasets or simulation-based experiments, which limit reproducibility, overlook hardware factors, and hinder fair human-robot comparisons. We present Switch-JustDance, a low-cost and reproducible benchmarking pipeline that leverages motion-sensing console games, Just Dance on the Nintendo Switch, to evaluate robot whole-body control. Using Just Dance on the Nintendo Switch as a representative platform, Switch-JustDance converts in-game choreography into robot-executable motions through streaming, motion reconstruction, and motion retargeting modules and enables users to evaluate controller performance through the game's built-in scoring system. We first validate the evaluation properties of Just Dance, analyzing its reliability, validity, sensitivity, and potential sources of bias. Our results show that the platform provides consistent and interpretable performance measures, making it a suitable tool for benchmarking embodied AI. Building on this foundation, we benchmark three state-of-the-art humanoid whole-body controllers on hardware and provide insights into their relative strengths and limitations.

Paper Structure

This paper contains 23 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Humanoid whole-body controllers
  4. Commercial games for AI benchmarking
  5. VR and console platforms for embodied AI
  6. Switch-JustDance
  7. System Validation
  8. Validity
  9. Discriminability
  10. Repeatability
  11. Potential Shortcoming
  12. Benchmark Metrics and Experiments
  13. Benchmark Metrics
  14. Experimental Settings
  15. Candidate Controllers
  16. ...and 8 more sections

Figures (3)

  • Figure 1: We introduce Switch-JustDance, a benchmark system for evaluating humanoid control policies using the Just Dance game on the Nintendo Switch. Using this system, we benchmark three general humanoid controllers: GMT, TWIST and Any2Track, and compare their performance against human players.
  • Figure 2: Switch-JustDance captures Nintendo Switch gameplay and streams it to a MoCap module that recovers the dancer’s motion in SMPL human motion. The pose is retargeted to the robot via the retarget module, executed by a whole-body controller, and the robot’s performance is scored in-game.
  • Figure 3: Top to bottom: Switch motion as source frames, retargeted motion from GMR output, and three humanoid controllers on hardware (TWIST, GMT, and Any2Track). Columns progress left to right in time. Unitree G1 plays Just Dance by holding a Joy-Con, enabling in-game scoring.