Table tennis ball spin estimation with an event camera

TL;DR

This work tackles the challenge of estimating table tennis ball spin under high-speed motion by leveraging a static event camera and the EROS time-surface representation. The authors implement a three-stage pipeline: real-time ball tracking with Kalman-filtered Hough-detected radius, extraction of events generated by the ball's logo, and spin estimation from logo optical flow (with a secondary event-rate based refinement). They demonstrate the approach on controlled ball-spinner and real ball-thrower setups, achieving real-time performance and competitive spin-magnitude and spin-axis accuracy, while highlighting limitations related to logo geometry and low-spin regimes. The study suggests that event-camera spin estimation can complement frame-based methods, with potential extensions to other sports and neural-network-based end-to-end spin estimation from events in the future.

Abstract

Spin plays a pivotal role in ball-based sports. Estimating spin becomes a key skill due to its impact on the ball's trajectory and bouncing behavior. Spin cannot be observed directly, making it inherently challenging to estimate. In table tennis, the combination of high velocity and spin renders traditional low frame rate cameras inadequate for quickly and accurately observing the ball's logo to estimate the spin due to the motion blur. Event cameras do not suffer as much from motion blur, thanks to their high temporal resolution. Moreover, the sparse nature of the event stream solves communication bandwidth limitations many frame cameras face. To the best of our knowledge, we present the first method for table tennis spin estimation using an event camera. We use ordinal time surfaces to track the ball and then isolate the events generated by the logo on the ball. Optical flow is then estimated from the extracted events to infer the ball's spin. We achieved a spin magnitude mean error of $10.7 \pm 17.3$ rps and a spin axis mean error of $32.9 \pm 38.2°$ in real time for a flying ball.

Figures (25)

• Figure 1: Comparison between the event stream of an event-based camera and captured frames from a frame-based camera. Black and blue dots are, respectively, ON- and OFF-events. The dotted red box highlights the events generated by the spinning logo.
• Figure 2: Pairs of representations for the same events. (left) EROS Gava2022arxiv event representations with $k_{eros}=10$. (right) Accumulated event frames with an accumulation time of $t_{acc}=2\text{ms}$.
• Figure 3: Tracked states of the Kalman filter: Positon, $\bm{p}_b = [x_b, y_b]$, velocity, $\bm{v}_b = [\dot{x}_b, \dot{y}_b]$ and ball radius $r$. The Kalman filter runs with $200$ Hz. The dots indicate measurements from the hough circle transform. The colored band around the estimated values represents the standard deviation.
• Figure 4: Sketch showing how the spin is calculated from the optical flow.
• Figure 5: Prediction of the spin magnitude from the event rate. The histograms of the event rate are displayed. We first apply an exponential moving average (EMA) and subtract the mean event rate. Then, we apply a low pass filter to smooth the curve. After that, we locate all the time stamps at which the sign of this curve transitions from positive to negative values.