Deep Learning for Sports Video Event Detection: Tasks, Datasets, Methods, and Challenges

TL;DR

The paper addresses the need for temporally precise sports video event detection by formalizing three core tasks—Temporal Action Localization (TAL), Action Spotting (AS), and Precise Event Spotting (PES)—and surveying deep learning approaches tailored to monocular broadcasts. It offers a taxonomy of temporal modeling strategies, multimodal fusion, and data-efficient pipelines, along with critical evaluation of benchmark datasets and metrics that often misalign with real-world requirements. Key contributions include clear task definitions, a structured methodological overview, and a discussion of open challenges such as cross-sport generalization, unsupervised learning, and deployment gaps. The work aims to bridge research and industry by guiding robust, frame-accurate, and practically deployable sports event detection systems for highlights, analytics, and health monitoring.

Abstract

Video event detection has become a cornerstone of modern sports analytics, powering automated performance evaluation, content generation, and tactical decision-making. Recent advances in deep learning have driven progress in related tasks such as Temporal Action Localization (TAL), which detects extended action segments; Action Spotting (AS), which identifies a representative timestamp; and Precise Event Spotting (PES), which pinpoints the exact frame of an event. Although closely connected, their subtle differences often blur the boundaries between them, leading to confusion in both research and practical applications. Furthermore, prior surveys either address generic video event detection or broader sports video tasks, but largely overlook the unique temporal granularity and domain-specific challenges of event spotting. In addition, most existing sports video surveys focus on elite-level competitions while neglecting the wider community of everyday practitioners. This survey addresses these gaps by: (i) clearly delineating TAL, AS, and PES and their respective use cases; (ii) introducing a structured taxonomy of state of the art approaches including temporal modeling strategies, multimodal frameworks, and data-efficient pipelines tailored for AS and PES; and (iii) critically assessing benchmark datasets and evaluation protocols, highlighting limitations such as reliance on broadcast quality footage and metrics that over reward permissive multilabel predictions. By synthesizing current research and exposing open challenges, this work provides a comprehensive foundation for developing temporally precise, generalizable, and practically deployable sports event detection systems for both the research and industry communities.

Figures (9)

• Figure 1: Annual publication count from 2010 to 2024 based on a Scopus keyword search for "sports" AND "deep learning".
• Figure 2: Example of Temporal Action Localization: in tennis, the full serve motion is annotated as a time interval (blue bar).
• Figure 3: Example of Precise Event Spotting: in table tennis, the moment a player contacts the ball during a serve (red) or when a ball bounces on the table (blue).
• Figure 4: Comparison of Global-to-Local (GTL, left) and Local-to-Global (LTG, right) approaches. GTL classifies predefined temporal anchors as action or background, followed by regression to refine time intervals. LTG predicts per-frame start and end probabilities to define action boundaries, which are then classified.
• Figure 5: Typical workflow of AS/PES models: an input video clip composed of multiple frames is first processed by a feature extractor (2D or 3D), followed by temporal modules to capture temporal dependencies. The final output can be frame-level predictions or clip-level classifications, depending on the task requirements.