Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Seg the HAB: Language-Guided Geospatial Algae Bloom Reasoning and Segmentation

Patterson Hsieh, Jerry Yeh, Mao-Chi He, Wen-Han Hsieh, Elvis Hsieh

TL;DR

ALGOS addresses the need for scalable HAB monitoring by jointly performing per-pixel segmentation and severity reasoning on wide-area remote-sensing data. It fuses a geospatial vision encoder with a language model in an embedding-as-mask framework and trains with a joint objective that couples text generation and mask prediction, using curated HAB-specific datasets via semi-supervised GeoSAM-based curation and synthetic severity queries. The approach yields strong segmentation (cIoU and gIoU) and severity (MSE) performance, outperforming state-of-the-art baselines and enabling actionable, language-guided ecological monitoring. The work advances automated cyanobacterial monitoring with practical implications for ecological management and public health, while noting limitations in geographic/seasonal generalization and data requirements for deployment.

Abstract

Climate change is intensifying the occurrence of harmful algal bloom (HAB), particularly cyanobacteria, which threaten aquatic ecosystems and human health through oxygen depletion, toxin release, and disruption of marine biodiversity. Traditional monitoring approaches, such as manual water sampling, remain labor-intensive and limited in spatial and temporal coverage. Recent advances in vision-language models (VLMs) for remote sensing have shown potential for scalable AI-driven solutions, yet challenges remain in reasoning over imagery and quantifying bloom severity. In this work, we introduce ALGae Observation and Segmentation (ALGOS), a segmentation-and-reasoning system for HAB monitoring that combines remote sensing image understanding with severity estimation. Our approach integrates GeoSAM-assisted human evaluation for high-quality segmentation mask curation and fine-tunes vision language model on severity prediction using the Cyanobacteria Aggregated Manual Labels (CAML) from NASA. Experiments demonstrate that ALGOS achieves robust performance on both segmentation and severity-level estimation, paving the way toward practical and automated cyanobacterial monitoring systems.

Seg the HAB: Language-Guided Geospatial Algae Bloom Reasoning and Segmentation

TL;DR

ALGOS addresses the need for scalable HAB monitoring by jointly performing per-pixel segmentation and severity reasoning on wide-area remote-sensing data. It fuses a geospatial vision encoder with a language model in an embedding-as-mask framework and trains with a joint objective that couples text generation and mask prediction, using curated HAB-specific datasets via semi-supervised GeoSAM-based curation and synthetic severity queries. The approach yields strong segmentation (cIoU and gIoU) and severity (MSE) performance, outperforming state-of-the-art baselines and enabling actionable, language-guided ecological monitoring. The work advances automated cyanobacterial monitoring with practical implications for ecological management and public health, while noting limitations in geographic/seasonal generalization and data requirements for deployment.

Abstract

Climate change is intensifying the occurrence of harmful algal bloom (HAB), particularly cyanobacteria, which threaten aquatic ecosystems and human health through oxygen depletion, toxin release, and disruption of marine biodiversity. Traditional monitoring approaches, such as manual water sampling, remain labor-intensive and limited in spatial and temporal coverage. Recent advances in vision-language models (VLMs) for remote sensing have shown potential for scalable AI-driven solutions, yet challenges remain in reasoning over imagery and quantifying bloom severity. In this work, we introduce ALGae Observation and Segmentation (ALGOS), a segmentation-and-reasoning system for HAB monitoring that combines remote sensing image understanding with severity estimation. Our approach integrates GeoSAM-assisted human evaluation for high-quality segmentation mask curation and fine-tunes vision language model on severity prediction using the Cyanobacteria Aggregated Manual Labels (CAML) from NASA. Experiments demonstrate that ALGOS achieves robust performance on both segmentation and severity-level estimation, paving the way toward practical and automated cyanobacterial monitoring systems.
Paper Structure (15 sections, 3 equations, 4 figures, 2 tables)

This paper contains 15 sections, 3 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Cyanobacteria Detection and Segmentation
  4. Geospatial Foundation Models
  5. Methods
  6. HAB Segmentation Dataset Curation
  7. HAB Reasoning Dataset Curation
  8. Vision-Language Model Architecture for HAB Monitoring
  9. Results
  10. Setup
  11. Results and Observations
  12. Conclusion
  13. Qualitative Comparison Across Models
  14. Image Segmentation
  15. Algae Severity Assessment

Figures (4)

  • Figure 1: The pipeline of ALGOS. Given the input image and text query, the multimodal LLM (LISAtpre) generates text output. The last-layer embedding for the <SEG> token is then decoded into the segmentation mask via the decoder. We adopt SAM kirillov2023sam as our choice of vision backbone.
  • Figure 2: Qualitative comparison of predictions across models.
  • Figure 3: Interactive segmentation workflow used during data curation.
  • Figure 4: Satellite image provided as input.