Visual Trajectory Prediction of Vessels for Inland Navigation

TL;DR

This paper addresses video-based vessel trajectory prediction for inland navigation by integrating state-of-the-art object detection, multi-object tracking, a Kalman filter, and spline-based interpolation. It evaluates several MOT baselines (BoT-SORT, Deep OC-SORT, ByeTrack) and demonstrates that the Kalman filter yields smoother trajectories and improved robustness to temporary detection gaps. It also highlights limitations of COCO-trained detectors in inland waterways, emphasizing the need for customized inland datasets and vessel classification to reduce false positives and improve safety-critical predictions. The work demonstrates practical relevance for remote operation and autonomous navigation, and outlines future work to expand datasets and incorporate vessel-type awareness for better collision avoidance and global-coordinate mapping.

Abstract

The future of inland navigation increasingly relies on autonomous systems and remote operations, emphasizing the need for accurate vessel trajectory prediction. This study addresses the challenges of video-based vessel tracking and prediction by integrating advanced object detection methods, Kalman filters, and spline-based interpolation. However, existing detection systems often misclassify objects in inland waterways due to complex surroundings. A comparative evaluation of tracking algorithms, including BoT-SORT, Deep OC-SORT, and ByeTrack, highlights the robustness of the Kalman filter in providing smoothed trajectories. Experimental results from diverse scenarios demonstrate improved accuracy in predicting vessel movements, which is essential for collision avoidance and situational awareness. The findings underline the necessity of customized datasets and models for inland navigation. Future work will expand the datasets and incorporate vessel classification to refine predictions, supporting both autonomous systems and human operators in complex environments.

Figures (12)

• Figure 1: Visualization of the parameters used for detection, tracking, and prediction on the example of a ship moving in the forward direction. The bounding box $\mathbf{b}_i^k$, the center of the box $\mathbf{q}_i^k$, the current anchor point $\mathbf{p}_i^k$ used for tracking, the trajectory of past frames (green) and the predicted trajectory (orange) with corresponding sample times for the current frame with the index $k$ and the object with the index $i$. The values $j_1>0$ and $j_2 > 0$ are used to illustrate the points in time.
• Figure 2: Relevance visualization of image areas for the assignment of objects to the boat class using a Faster R-CNN detector trained on the Microsoft Common Objects in Context (COCO) data set COCO. Based on the result values of the Score-CAM ScoreCAM method, the overlay of the respective original image with red indicates a higher relevance, with yellow a medium relevance and with blue a lower relevance. The three images show consecutive frames from the same video where vessels are moving in the forward direction. Note the slight camera movement, which complicates tracking due to a changed perspective.
• Figure 3: The class activation map presents a high confidence for many objects at the shoreline and a low confidence for the center section of the vessels. Furthermore, longer vessels are represented by multiple high-confidence values.
• Figure 4: The class activation maps present a high level of confidence for the fishing rod and the island in the background.
• Figure 5: Comparison of detection quality between inland and sea-going vessels