Survey of Action Recognition, Spotting and Spatio-Temporal Localization in Soccer -- Current Trends and Research Perspectives
Karolina Seweryn, Anna Wróblewska, Szymon Łukasik
TL;DR
This survey systematically catalogs the state of soccer action understanding across three tasks: action recognition, spotting, and spatio-temporal localization. It emphasizes multimodal learning and the integration of video, audio, text, and pose/graph data, reviewing datasets (notably SoccerNet families and MultiSports) and a broad array of methods from classical feature-based approaches to transformer- and graph-based models. Key metrics such as $mAP$, $avg$-$mAP$, and $3D$ $IoU$ are detailed for evaluating performance, alongside motion-aware extensions. The paper highlights substantial progress driven by richer representations and public datasets, while identifying ongoing challenges in data availability, annotation quality, and the development of robust soccer-specific localization benchmarks. It further argues that multimodal data and cross-domain datasets hold substantial promise for advancing real-world soccer analytics, highlighting future directions such as text-informed action understanding and more comprehensive spatio-temporal localization datasets.
Abstract
Action scene understanding in soccer is a challenging task due to the complex and dynamic nature of the game, as well as the interactions between players. This article provides a comprehensive overview of this task divided into action recognition, spotting, and spatio-temporal action localization, with a particular emphasis on the modalities used and multimodal methods. We explore the publicly available data sources and metrics used to evaluate models' performance. The article reviews recent state-of-the-art methods that leverage deep learning techniques and traditional methods. We focus on multimodal methods, which integrate information from multiple sources, such as video and audio data, and also those that represent one source in various ways. The advantages and limitations of methods are discussed, along with their potential for improving the accuracy and robustness of models. Finally, the article highlights some of the open research questions and future directions in the field of soccer action recognition, including the potential for multimodal methods to advance this field. Overall, this survey provides a valuable resource for researchers interested in the field of action scene understanding in soccer.