Towards Ball Spin and Trajectory Analysis in Table Tennis Broadcast Videos via Physically Grounded Synthetic-to-Real Transfer

TL;DR

This work tackles the challenge of extracting the ball's 3D trajectory and initial spin from monocular table tennis broadcasts by training entirely on physics-grounded synthetic data. It introduces a Spin Prediction Transformer that ingests a compact representation—2D ball trajectories plus 13 table keypoints—and predicts both $\,vec{r}(t_i)$ and $\,vec{\omega}$. Through physics-based data generation and targeted augmentations, the approach achieves strong real-world generalization, attaining $acc = 92.0\%$ for spin classification and a 2D reprojection error of $0.19\%$ of the image diagonal. This enables detailed spin and trajectory analysis directly from standard broadcast videos, with potential impact for broadcast analytics and accessible performance evaluation.

Abstract

Analyzing a player's technique in table tennis requires knowledge of the ball's 3D trajectory and spin. While, the spin is not directly observable in standard broadcasting videos, we show that it can be inferred from the ball's trajectory in the video. We present a novel method to infer the initial spin and 3D trajectory from the corresponding 2D trajectory in a video. Without ground truth labels for broadcast videos, we train a neural network solely on synthetic data. Due to the choice of our input data representation, physically correct synthetic training data, and using targeted augmentations, the network naturally generalizes to real data. Notably, these simple techniques are sufficient to achieve generalization. No real data at all is required for training. To the best of our knowledge, we are the first to present a method for spin and trajectory prediction in simple monocular broadcast videos, achieving an accuracy of 92.0% in spin classification and a 2D reprojection error of 0.19% of the image diagonal.

Figures (10)

• Figure 1: Simulated trajectory of the ball in the image plane under the influence of different spin components $\omega_{\tilde{x}}$, $\omega_{\tilde{y}}$, and $\omega_{\tilde{z}}$.
• Figure 2: Overview of our pipeline. For each time step $t_i$, ball coordinates and table keypoints are embedded to generate location tokens $l_i$. A learnable spin token $s$ is prepended, and the SPT processes the sequence $\{s, l_0, ..., l_{T-1}\}$. The SPT predicts the initial spin $\vec{\omega}$ at $t_0$ and the sequence of 3D ball positions $\{\vec{r}(t_0), ..., \vec{r}(t_{T-1})\}$. The predicted spin and trajectory are supervised using separate loss terms, ensuring accurate learning of both components.
• Figure 3: World coordinate system ($x$, $y$, $z$) and ball coordinate system ($\tilde{x}$, $\tilde{y}$, $\tilde{z}$). Both are orthogonal coordinate systems.
• Figure 4: Illustration of the 3 different SPT architectures. In the single-stage model, the trajectory is predicted jointly with the spin. The two-stage model predicts only the 3D positions $\vec{r}(t_i) \in \mathcal{R}^3$ in the first stage and uses these predictions to estimate the spin in the second stage. The connect-stage model uses the transformed tokens $l_i \in \mathcal{R}^d$ as input for the second stage.
• Figure 5: Confusion matrix and ROC plot for the best model on the real dataset.