Neural Network-Based Tracking and 3D Reconstruction of Baseball Pitch Trajectories from Single-View 2D Video
Jhen Hsieh
TL;DR
The paper addresses tracking and reconstructing baseball pitch trajectories in 3D from a single 2D video stream. It proposes a neural-network framework that uses OpenCV CSRT to track the ball and fixed references, feeding 2D pixel coordinates along with timestamp information into a multi-layer network trained with mean squared error to predict 3D positions. Synthetic data generation projects simulated trajectories onto a screen with fixed references and timestamps, enabling supervised learning without multi-camera rigs. Experimental results claim high accuracy and emphasize the method’s accessibility, reduced hardware requirements, and potential applicability to other sports.
Abstract
In this paper, we present a neural network-based approach for tracking and reconstructing the trajectories of baseball pitches from 2D video footage to 3D coordinates. We utilize OpenCV's CSRT algorithm to accurately track the baseball and fixed reference points in 2D video frames. These tracked pixel coordinates are then used as input features for our neural network model, which comprises multiple fully connected layers to map the 2D coordinates to 3D space. The model is trained on a dataset of labeled trajectories using a mean squared error loss function and the Adam optimizer, optimizing the network to minimize prediction errors. Our experimental results demonstrate that this approach achieves high accuracy in reconstructing 3D trajectories from 2D inputs. This method shows great potential for applications in sports analysis, coaching, and enhancing the accuracy of trajectory predictions in various sports.