eSkiTB: A Synthetic Event-based Dataset for Tracking Skiers

TL;DR

This paper introduces eSkiTB, a synthetic event-based ski-tracking dataset generated from SkiTB under strict iso-informational constraints to enable fair comparisons between RGB and event modalities. By converting RGB frames to event streams without neural interpolation, the dataset preserves true temporal dynamics and provides a platform for evaluating spiking trackers on high-speed, cluttered broadcasts. The results show that finetuned SDTrack on eSkiTB achieves a mean IoU of $0.711$, outperforming pretrained baselines and demonstrating robustness to broadcast clutter, with a notable IoU of $0.685$ in cluttered scenarios. Overall, the work highlights the potential of temporal-contrast sensing for winter-sport tracking and provides a valuable benchmark to drive adaptive event representations and domain-specific neuromorphic trackers.

Abstract

Tracking skiers in RGB broadcast footage is challenging due to motion blur, static overlays, and clutter that obscure the fast-moving athlete. Event cameras, with their asynchronous contrast sensing, offer natural robustness to such artifacts, yet a controlled benchmark for winter-sport tracking has been missing. We introduce event SkiTB (eSkiTB), a synthetic event-based ski tracking dataset generated from SkiTB using direct video-to-event conversion without neural interpolation, enabling an iso-informational comparison between RGB and event modalities. Benchmarking SDTrack (spiking transformer) against STARK (RGB transformer), we find that event-based tracking is substantially resilient to broadcast clutter in scenes dominated by static overlays, achieving 0.685 IoU, outperforming RGB by +20.0 points. Across the dataset, SDTrack attains a mean IoU of 0.711, demonstrating that temporal contrast is a reliable cue for tracking ballistic motion in visually congested environments. eSkiTB establishes the first controlled setting for event-based tracking in winter sports and highlights the promise of event cameras for ski tracking. The dataset and code will be released at https://github.com/eventbasedvision/eSkiTB.

Figures (9)

• Figure 1: Represents a visual overview of the eSkiTB dataset. The RGB frame illustrates the challenge of background clutter and broadcast artifacts, while the corresponding event stream demonstrates the natural filtering of static distractions like text overlays and fences.
• Figure 2: Illustrates the weakness in standard RGB images which suffers from screen-space artifacts (yellow) and world-space clutter (red). While these artifacts obscure the athlete in the intensity domain, our proposed event-based pipeline naturally filters the static screen-space overlays.
• Figure 3: Qualitative comparison on (a) sample RGB ski-jump, (b) v2e preserves the skier's silhouette and motion details better than (c) ESIM and (d) Prophesee's simulator, which exhibits significant noise and artifacts.
• Figure 4: Illustrates eSkiTB generation pipeline. We process raw high-resolution RGB frames through the v2e simulator to generate asynchronous event streams. Crucially, we bypass neural frame interpolation to maintain an iso-informational constraint, ensuring that the event data contains no hallucinated temporal information. Ground truth bounding boxes are interpolated to $1$ ms resolution.
• Figure 5: Represents eSkiTB Statistics. (a) Density: Heavy-tailed global rates (95th percentile: $1.58$, Meps) confirm high-dynamic motion. (b) SNR: Consistent shift to higher target densities proves natural segmentation from background clutter. (c) Scale: Skewed bbox area ($< 4\%$ of image) reflects extreme zoom-induced variance. (d) Temporal: Information density peaks during critical take-off ($\approx 20\%$) and flight.