SoccerSynth Field: enhancing field detection with synthetic data from virtual soccer simulator

TL;DR

The paper tackles the cost and practicality of collecting large, diverse real-world data for soccer field detection by leveraging synthetic data. It introduces SoccerSynth-Field, a simulator-based dataset generated with Unreal Engine 5 that enables controlled variations in lighting, camera angles, ground patterns, and even fake field lines to challenge models. A DeepLabV3-ResNet-101 segmentation model is pretrained on synthetic data and finetuned on real SoccerNet-Calibration data, then evaluated on WC14, with results showing synthetic pretraining can match or exceed real-data baselines, particularly in low-data scenarios. The findings support synthetic data as a cost-effective, scalable augmentation to real-world datasets for robust field-detection in broadcast soccer videos, with potential for further environmental variation enhancements.

Abstract

Field detection in team sports is an essential task in sports video analysis. However, collecting large-scale and diverse real-world datasets for training detection models is often cost and time-consuming. Synthetic datasets, which allow controlled variability in lighting, textures, and camera angles, will be a promising alternative for addressing these problems. This study addresses the challenges of high costs and difficulties in collecting real-world datasets by investigating the effectiveness of pretraining models using synthetic datasets. In this paper, we propose the effectiveness of using a synthetic dataset (SoccerSynth-Field) for soccer field detection. A synthetic soccer field dataset was created to pretrain models, and the performance of these models was compared with models trained on real-world datasets. The results demonstrate that models pretrained on the synthetic dataset exhibit superior performance in detecting soccer fields. This highlights the effectiveness of synthetic data in enhancing model robustness and accuracy, offering a cost-effective and scalable solution for advancing detection tasks in sports field detection.

Figures (5)

• Figure 1: Keypoints in our simulator: We placed numerous keypoints along the soccer field lines, naming them accordingly. During screenshots, we preserved the names and coordinates of these keypoints to reconstruct their trajectories. We then label spceific keypoints for training the model.
• Figure 2: Tags of field. This figure illustrates the annotations of the soccer field such as side lines, goal areas, and center circles. The purpose is to provide a detailed reference for analyzing field elements in field detection.
• Figure 3: Comparison of synthetic datasets for soccer field detection: The upper image is JA dataset (audience and jersey randomizations only), which lacks complex environmental factors. The lower image is SoccerSynth-Field (JA+G+P+L+CM+F) dataset, added elements such as lighting, ground patterns, and audiences, enhanced diversity and realism to improve model robustness and generalization to real-world conditions.
• Figure 4: Comparison of field detection performance in JA (upper) and SoccerSynth-Field (lower) dataset without finetuning. Although the JA shows frequent misdetections, the SoccerSynth-Field demonstrates higher accuracy due to its exposure to diverse randomization elements during training.
• Figure 5: Comparison of baseline (real-world data only) and SoccerSynth-Field(with pretraining). Both of them achieve comparable detection accuracy, even in scenes with noise and occlusions. Moreover, the SoccerSynth-Field model excels in handling finer details, as exemplified in the bottom-right corner of the figure, where it avoids the errors observed in the baseline model, indicating that synthetic data pretraining can effectively complement real-world data without causing noticeable degradation in results.