Enhancing Maritime Situational Awareness through End-to-End Onboard Raw Data Analysis

TL;DR

The feasibility of the proposed method is demonstrated through a proof-of-concept on CubeSat-like hardware, confirming the models' potential for operational satellite-based maritime monitoring.

Abstract

Satellite-based onboard data processing is crucial for time-sensitive applications requiring timely and efficient rapid response. Advances in edge artificial intelligence are shifting computational power from ground-based centers to on-orbit platforms, transforming the "sensing-communication-decision-feedback" cycle and reducing latency from acquisition to delivery. The current research presents a framework addressing the strict bandwidth, energy, and latency constraints of small satellites, focusing on maritime monitoring. The study contributes three main innovations. Firstly, it investigates the application of deep learning techniques for direct ship detection and classification from raw satellite imagery. By simplifying the onboard processing chain, our approach facilitates direct analyses without requiring computationally intensive steps such as calibration and ortho-rectification. Secondly, to address the scarcity of raw satellite data, we introduce two novel datasets, VDS2Raw and VDV2Raw, which are derived from raw data from Sentinel-2 and Vegetation and Environment Monitoring New Micro Satellite (VENuS) missions, respectively, and enriched with Automatic Identification System (AIS) records. Thirdly, we characterize the tasks' optimal single and multiple spectral band combinations through statistical and feature-based analyses validated on both datasets. In sum, we demonstrate the feasibility of the proposed method through a proof-of-concept on CubeSat-like hardware, confirming the models' potential for operational satellite-based maritime monitoring.

Figures (14)

• Figure 1: Graphical comparison of the normalized spectral responses between the on the VENµS satellite (top) and the (bottom) highlighting key differences in their spectral band coverage. Specifically, the VENµS bands $B_i$ ($i=1,5,6,12$) do not have corresponding bands in the imager. Conversely, bands $B_8$ and $B_9$ of are absent in VENµS, indicating that each is optimized for different spectral features.
• Figure 2: (a) Unequalized VENµS image of the Ebro Delta, showing significant stray light artifacts in band $B_2$, which affects the visibility of underlying features. (b) Striping noise observed in band $B_2$ of the VENµS image captured over the port of Ashdod, illustrating the impact of lack of calibration on coastal imagery. (c) Radiometric noise present in band $B_3$ of the image over an algal bloom area, highlighting the challenges of detecting subtle biological features amidst sensor noise. (d) Radiometric noise detected in band $B_3$ of the image over the Danish Fjord, demonstrating the effects of sensor-induced noise on the accurate interpretation of aquatic environments.
• Figure 3: Bounding box fitting of a label in the $B_6$ band (VENµS ). Image gathered over the port of Ashdod on 2020/01/06. The vessel, identified by 636019532, is a Container Ship with a length of 159.8 meters and a width of 24.8 meters. The ship was located at a longitude of 34.58744 and a latitude of 31.8442 at the time of the capture.
• Figure 4: Graphical representation of the geographical coverage by the and VENµS datasets (a). The histograms illustrate the distribution of bounding boxes area and aspect ratio, offering a clear view of the training, validation, and test datasets for (b) and VENµS (c) missions.
• Figure 5: The AIS characteristics of the raw vessel datasets analyzed through various statistical visualizations. Histograms and kernel density estimates of length and width distributions are presented across training, validation, and test subsets for both the VDS2Raw (a) and VDVRaw (b) datasets. Additionally, bar plots (c) and (d) illustrate the frequency distribution of each vessel category within the VDS2Raw and VDVRaw datasets, respectively.