Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CyboRacket: A Perception-to-Action Framework for Humanoid Racket Sports

Peng Ren, Chuan Qi, Haoyang Ge, Qiyuan Su, Xuguo He, Cong Huang, Pei Chi, Jiang Zhao, Kai Chen

Abstract

Dynamic ball-interaction tasks remain challenging for robots because they require tight perception-action coupling under limited reaction time. This challenge is especially pronounced in humanoid racket sports, where successful interception depends on accurate visual tracking, trajectory prediction, coordinated stepping, and stable whole-body striking. Existing robotic racket-sport systems often rely on external motion capture for state estimation or on task-specific low-level controllers that must be retrained across tasks and platforms. We present CyboRacket, a hierarchical perception-to-action framework for humanoid racket sports that integrates onboard visual perception, physics-based trajectory prediction, and large-scale pre-trained whole-body control. The framework uses onboard cameras to track the incoming object, predicts its future trajectory, and converts the estimated interception state into target end-effector and base-motion commands for whole-body execution by SONIC on the Unitree G1 humanoid robot. We evaluate the proposed framework in a vision-based humanoid tennis-hitting task. Experimental results demonstrate real-time visual tracking, trajectory prediction, and successful striking using purely onboard sensing.

CyboRacket: A Perception-to-Action Framework for Humanoid Racket Sports

Abstract

Dynamic ball-interaction tasks remain challenging for robots because they require tight perception-action coupling under limited reaction time. This challenge is especially pronounced in humanoid racket sports, where successful interception depends on accurate visual tracking, trajectory prediction, coordinated stepping, and stable whole-body striking. Existing robotic racket-sport systems often rely on external motion capture for state estimation or on task-specific low-level controllers that must be retrained across tasks and platforms. We present CyboRacket, a hierarchical perception-to-action framework for humanoid racket sports that integrates onboard visual perception, physics-based trajectory prediction, and large-scale pre-trained whole-body control. The framework uses onboard cameras to track the incoming object, predicts its future trajectory, and converts the estimated interception state into target end-effector and base-motion commands for whole-body execution by SONIC on the Unitree G1 humanoid robot. We evaluate the proposed framework in a vision-based humanoid tennis-hitting task. Experimental results demonstrate real-time visual tracking, trajectory prediction, and successful striking using purely onboard sensing.
Paper Structure (19 sections, 23 equations, 4 figures)

This paper contains 19 sections, 23 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Robotic Racket Sports and Dynamic Object Interaction
  4. Humanoid Whole-Body Control and Scalable Motion Tracking
  5. System Overview
  6. Onboard RGB-D projectile Perception and State Estimation
  7. Projectile Observation and 2D Tracking
  8. RGB-D 3D Localization and Measurement Uncertainty
  9. EKF-Based projectile State Estimation
  10. Short-Horizon Trajectory Prediction
  11. Interception Planning and Pre-trained Whole-Body Control
  12. Interception Planning
  13. Online Replanning and Swing Coordination
  14. Task-Space Target Generation
  15. Execution Interface to SONIC
  16. ...and 4 more sections

Figures (4)

  • Figure 1: System overview of the humanoid humanoid racket sports framework.
  • Figure 2: Representative motion sequence of humanoid badminton hitting in simulation.
  • Figure 3: Representative sequence of vision-based tennis hitting on the humanoid robot.
  • Figure 4: Representative sequence of vision-based badminton hitting on the humanoid robot.