Vessel Re-identification and Activity Detection in Thermal Domain for Maritime Surveillance

TL;DR

This work tackles vessel re-identification and suspicious-at-sea activity detection under nighttime conditions by introducing a thermal-domain maritime surveillance pipeline. It combines three subsystems—TraDeS-based object tracking (thermal adaptation), a viewpoint-independent vessel re-identification module using separate side-view latent spaces with ArcFace losses, and an adapted YOWO-based spatiotemporal activity detector—along with a foreground-encoder-decoder for robust vessel masking. A newly created public thermal maritime dataset supports evaluation, and the proposed re-identification approach achieves a Top1 of $81.8\%$ and frame-level mAP of $72.4\%$ in the thermal domain, outperforming RGB/IR baselines like SPAN. The integrated system demonstrates competitive thermal-domain performance and establishes a practical foundation for night-time maritime surveillance, with future work targeting higher frame rates and hardware optimization.

Abstract

Maritime surveillance is vital to mitigate illegal activities such as drug smuggling, illegal fishing, and human trafficking. Vision-based maritime surveillance is challenging mainly due to visibility issues at night, which results in failures in re-identifying vessels and detecting suspicious activities. In this paper, we introduce a thermal, vision-based approach for maritime surveillance with object tracking, vessel re-identification, and suspicious activity detection capabilities. For vessel re-identification, we propose a novel viewpoint-independent algorithm which compares features of the sides of the vessel separately (separate side-spaces) leveraging shape information in the absence of color features. We propose techniques to adapt tracking and activity detection algorithms for the thermal domain and train them using a thermal dataset we created. This dataset will be the first publicly available benchmark dataset for thermal maritime surveillance. Our system is capable of re-identifying vessels with an 81.8% Top1 score and identifying suspicious activities with a 72.4\% frame mAP score; a new benchmark for each task in the thermal domain.

Figures (8)

• Figure 1: Overall structure of the proposed surveillance system. Object detection and tracking algorithms (A) identify maritime objects in each frame, and crop and feed them to the re-identification algorithm (B). It extracts features from each visible side of the vessel and compares them with the dynamic database. Activity detection algorithm (C) detects suspicious activities. Sections 3.1, 3.2, and 3.3 further discuss (A), (B), and (C) subsystems, respectively.
• Figure 2: Object tracking model architecture
• Figure 3: (a) Results of the encoder-decoder model used in foreground extracting. (b) Masks generated by SPAN for front, side and rear views using the foreground mask.
• Figure 4: Re-identification subsystem. (a) extracts features using the vision transformer and maps them to four latent spaces. (b) calculates the area ratios of each visual side of the vessel. (c) and (d) calculate triplet loss and the ID loss, respectively.
• Figure 5: Mapping visual orientations to numbers: calculating the area ratio vector of the Fig. \ref{['fig_reid_arch']}.(b). The algorithm accurately identifies visible sides and assigns accurate values considering the area ratios.