Fusing Multi- and Hyperspectral Satellite Data for Harmful Algal Bloom Monitoring with Self-Supervised and Hierarchical Deep Learning

TL;DR

This work introduces SIT-FUSE, a self-supervised, multi-sensor data fusion framework for detecting and mapping harmful algal bloom severity and speciation across coastal regions. By integrating ocean-color reflectances from multiple satellites with TROPOSIF SIF measurements and employing hierarchical deep clustering, the approach produces context-aware phytoplankton concentration maps without requiring instrument-specific labels. Validation against in-situ data in the Gulf of Mexico and Southern California demonstrates the method’s potential to deliver scalable HAB monitoring in label-scarce environments, with promising results for extending to hyperspectral inputs like PACE OCI. The framework’s modular SSL encoders, data fusion strategies, and hierarchical embeddings enable exploratory analysis, cross-instrument tracking, and gradual operationalization for global aquatic biogeochemistry.

Abstract

We present a self-supervised machine learning framework for detecting and mapping harmful algal bloom (HAB) severity and speciation using multi-sensor satellite data. By fusing reflectance data from operational instruments (VIIRS, MODIS, Sentinel-3, PACE) with TROPOMI solar-induced fluorescence (SIF), our framework, called SIT-FUSE, generates HAB severity and speciation products without requiring per-instrument labeled datasets. The framework employs self-supervised representation learning, hierarchical deep clustering to segment phytoplankton concentrations and speciations into interpretable classes, validated against in-situ data from the Gulf of Mexico and Southern California (2018-2025). Results show strong agreement with total phytoplankton, Karenia brevis, Alexandrium spp., and Pseudo-nitzschia spp. measurements. This work advances scalable HAB monitoring in label-scarce environments while enabling exploratory analysis via hierarchical embeddings: a critical step toward operationalizing self-supervised learning for global aquatic biogeochemistry.

Figures (14)

• Figure 1: A flow diagram for the processing of one input type (single instrument or fusion set) through SIT-FUSE.
• Figure 2: A 2-layer example of the setup for hierarchical deep clustering. Each box labeled ‘Cluster’ is a set of fully connected layers, connected to the encoder and trained via the IIC loss function. Each child node is only trained and makes predictions on samples given the label from its parent nodes. This setup allows us to use deep clustering to create interlaced levels of specificity for data exploration and characterization.
• Figure 3: A depiction of all of the locations where in situ data was collected and used for context assignment and validation: a) West Florida cases in 2018-2019, b) Florida 2024-2025, and c) Southern CA in the 2018-2019 and 2024-2025 cases. The actual process of context assignment is done by generating simple histograms, or counts of overlap between a specific index within the binned set of phytoplankton concentrations and each label in the context-free segmentation products. The final assignment is done by assigning a context-free label to the phytoplankton concentration bin that it most frequently overlapped with. More sophisticated thresholding and overall assignment techniques can be applied, but we found this simple approach to be suitable for the cases tested.
• Figure 4: A depiction of the multi-tiered context assignment process based on the hierarchical context-free segmentation products. Step 1 (top) consists of finding the context-free labels that best match with different binned phytoplankton or speciated HAB concentration levels. Once this is done, the first context assignment is done accordingly. Next, Step 2 (bottom) consists of applying the same process to the Layer-2 context-free segmentation product, and then supplementing the agreement computations there by also looking at agreement between the Layer-2 context-free labels and the concentration labels assigned in step 1 over the scene. This process is done collectively over the set of scenes in the training set and Step 2 provides the final context assignment to be used for all scenes.
• Figure 5: A depiction of the combination of the various data streams. First OC only, TROPOSIF only, and TROPOSIF + OC concentration datasets are combined into a single phytoplankton or speciated HAB concentration product. An associated Data Quality Indicator (DQI) product is also generated, denoting from which data stream a given pixel came from. Lastly, monthly averages were generated for both concentration and DQI.