Enhancing Soccer Camera Calibration Through Keypoint Exploitation

TL;DR

This paper introduces a multi-stage pipeline that addresses this challenge of accurate camera calibration by leveraging the structural features of the football pitch by exploiting line-line and line-conic intersections, points on the conics, and other geometric features.

Abstract

Accurate camera calibration is essential for transforming 2D images from camera sensors into 3D world coordinates, enabling precise scene geometry interpretation and supporting sports analytics tasks such as player tracking, offside detection, and performance analysis. However, obtaining a sufficient number of high-quality point pairs remains a significant challenge for both traditional and deep learning-based calibration methods. This paper introduces a multi-stage pipeline that addresses this challenge by leveraging the structural features of the football pitch. Our approach significantly increases the number of usable points for calibration by exploiting line-line and line-conic intersections, points on the conics, and other geometric features. To mitigate the impact of imperfect annotations, we employ data fitting techniques. Our pipeline utilizes deep learning for keypoint and line detection and incorporates geometric constraints based on real-world pitch dimensions. A voter algorithm iteratively selects the most reliable keypoints, further enhancing calibration accuracy. We evaluated our approach on the largest football broadcast camera calibration dataset available, and secured the top position in the SoccerNet Camera Calibration Challenge 2023 [arXiv:2309.06006], which demonstrates the effectiveness of our method in real-world scenarios. The project code is available at https://github.com/NikolasEnt/soccernet-calibration-sportlight .

Figures (4)

• Figure 1: Keypoints used as the points detection model target. Red - points of line-line intersections; Blue - line-conic intersection points; Purple - conic tangent points; Black - other points projected by homography.
• Figure 2: Top row (left): Red dots denote the central circle points from the annotation. The red curve represents the ellipse fit to these points. Blue crosses indicate computed line-conic intersections, the red cross marks a computed line-line intersection, and purple crosses represent the derived tangent points to the ellipse. Note that the intersection points (red and blue crosses) are insufficient for homography computation. The corresponding points on the pitch pattern are shown on the right. Bottom row: Heatmaps of all target points overlaid on the video frame.
• Figure 3: Original image with a magenta line overlay for 'Goal left post left' line (top) and the corresponding heatmap channel of the line detection target (bottom).
• Figure 4: The keypoint detection model architecture. The line model follows the same structure but differs in the number of target featuremaps.