Whole Body Model Predictive Control for Spin-Aware Quadrupedal Table Tennis

TL;DR

This work addresses the challenge of playing dynamic table tennis with a quadrupedal robot by integrating fast, camera-based ball localization, spin-aware trajectory prediction, and a whole-body model predictive control framework. A Bezier-based kinematic planner coupled with a whole-body QP controller enables agile, spin-controlled swings, while a residual-enhanced trajectory predictor and a fast SQP-based aiming module generate effective strike plans. Hardware experiments on a Spot robot demonstrate high-accuracy perception, robust spin handling (incoming up to $|oldsymbol{c9}^-|$ up to hundreds of rad/s) and outgoing spin up to around $200$ rad/s, with a mean system return rate of about 90% and human-in-the-loop rallies. The results highlight the practical potential of integrated perception-prediction-control for legged robots in fast-paced racket sports, while also outlining avenues for onboard perception, stepping capabilities, and more adaptive strategy development.

Abstract

Developing table tennis robots that mirror human speed, accuracy, and ability to predict and respond to the full range of ball spins remains a significant challenge for legged robots. To demonstrate these capabilities we present a system to play dynamic table tennis for quadrupedal robots that integrates high speed perception, trajectory prediction, and agile control. Our system uses external cameras for high-speed ball localization, physical models with learned residuals to infer spin and predict trajectories, and a novel model predictive control (MPC) formulation for agile full-body control. Notably, a continuous set of stroke strategies emerge automatically from different ball return objectives using this control paradigm. We demonstrate our system in the real world on a Spot quadruped, evaluate accuracy of each system component, and exhibit coordination through the system's ability to aim and return balls with varying spin types. As a further demonstration, the system is able to rally with human players.

Figures (10)

• Figure 1: Dynamic whole-body quadruped swinging. Robot states are shown in gray with the paddle trajectory in blue. All renderings are generated using Viser viser
• Figure 2: System diagram with RGB cameras shown as wire frame pyramids that detect the ball in orange. Its predicted trajectory is shown in green with the strike plane in blue. The black motion capture cameras, located in the background, observe the position of the robot. The target ball landing location is in red on the table.
• Figure 3: Spin predicton performance using the model-based estimate on the left and the residual network on the right.
• Figure 4: Convergence of aiming planner to a given $\mathbf{p}_\text{land}$, $\omega^+$, and $t_\text{land}$. The paddle orientation and velocity is shown in red and purple respectively. The simulated resulting trajectory is shown in orange and $\mathbf{p}_\text{land}$ is indicated with the green point. The lighter colored components represent intermediate solutions during the SQP iterations.
• Figure 5: Variety of swing types including loop (top spin), drive (no spin), and chop (back spin).