Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing kelp forest detection in remote sensing images using crowdsourced labels with Mixed Vision Transformers and ConvNeXt segmentation models

Ioannis Nasios

TL;DR

The study tackles automated detection of kelp canopy in Landsat imagery, addressing label noise by leveraging crowdsourced annotations. It combines crowdsourced ground truth with a mixed architecture ensemble of MIT-based U-Net and ConvNeXt with UpperNet, trained on multiple image sizes and enhanced by postprocessing using altimetry-derived land-sea masks. The approach yields high per-pixel accuracy, achieving strong Dice scores and ranking third in a competitive semantic segmentation challenge, with notable gains from test-time augmentation and threshold tuning. This work demonstrates a scalable, cost-effective pathway for long-term coastal ecosystem monitoring using publicly available satellite data and crowdsourced labels, with code available for reproducibility and extension to other regions.

Abstract

Kelp forests, as foundation species, are vital to marine ecosystems, providing essential food and habitat for numerous organisms. This study explores the integration of crowdsourced labels with advanced artificial intelligence models to develop a fast and accurate kelp canopy detection pipeline using Landsat images. Building on the success of a machine learning competition, where this approach ranked third and performed consistently well on both local validation and public and private leaderboards, the research highlights the effectiveness of combining Mixed Vision Transformers (MIT) with ConvNeXt models. Training these models on various image sizes significantly enhanced the accuracy of the ensemble results. U-Net emerged as the best segmentation architecture, with UpperNet also contributing to the final ensemble. Key Landsat bands, such as ShortWave InfraRed (SWIR1) and Near-InfraRed (NIR), were crucial while altitude data was used in postprocessing to eliminate false positives on land. The methodology achieved a high detection rate, accurately identifying about three out of four pixels containing kelp canopy while keeping false positives low. Despite the medium resolution of Landsat satellites, their extensive historical coverage makes them effective for studying kelp forests. This work also underscores the potential of combining machine learning models with crowdsourced data for effective and scalable environmental monitoring. All running code for training all models and inference can be found at https://github.com/IoannisNasios/Kelp_Forests.

Enhancing kelp forest detection in remote sensing images using crowdsourced labels with Mixed Vision Transformers and ConvNeXt segmentation models

TL;DR

The study tackles automated detection of kelp canopy in Landsat imagery, addressing label noise by leveraging crowdsourced annotations. It combines crowdsourced ground truth with a mixed architecture ensemble of MIT-based U-Net and ConvNeXt with UpperNet, trained on multiple image sizes and enhanced by postprocessing using altimetry-derived land-sea masks. The approach yields high per-pixel accuracy, achieving strong Dice scores and ranking third in a competitive semantic segmentation challenge, with notable gains from test-time augmentation and threshold tuning. This work demonstrates a scalable, cost-effective pathway for long-term coastal ecosystem monitoring using publicly available satellite data and crowdsourced labels, with code available for reproducibility and extension to other regions.

Abstract

Kelp forests, as foundation species, are vital to marine ecosystems, providing essential food and habitat for numerous organisms. This study explores the integration of crowdsourced labels with advanced artificial intelligence models to develop a fast and accurate kelp canopy detection pipeline using Landsat images. Building on the success of a machine learning competition, where this approach ranked third and performed consistently well on both local validation and public and private leaderboards, the research highlights the effectiveness of combining Mixed Vision Transformers (MIT) with ConvNeXt models. Training these models on various image sizes significantly enhanced the accuracy of the ensemble results. U-Net emerged as the best segmentation architecture, with UpperNet also contributing to the final ensemble. Key Landsat bands, such as ShortWave InfraRed (SWIR1) and Near-InfraRed (NIR), were crucial while altitude data was used in postprocessing to eliminate false positives on land. The methodology achieved a high detection rate, accurately identifying about three out of four pixels containing kelp canopy while keeping false positives low. Despite the medium resolution of Landsat satellites, their extensive historical coverage makes them effective for studying kelp forests. This work also underscores the potential of combining machine learning models with crowdsourced data for effective and scalable environmental monitoring. All running code for training all models and inference can be found at https://github.com/IoannisNasios/Kelp_Forests.
Paper Structure (15 sections, 2 equations, 4 figures, 8 tables)

This paper contains 15 sections, 2 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Data
  4. Annotation errors
  5. Metric
  6. Modelling
  7. Data used and preprocessing
  8. Models
  9. Training parameters and augmentation
  10. Validation scheme
  11. Final predictions
  12. Results and discussion
  13. Insights
  14. Perspectives
  15. Conclusions

Figures (4)

  • Figure 1: Falkland Islands
  • Figure 2: Annotation error, chip WU193724
  • Figure 3: Shifted mask, chip QI166183
  • Figure 4: Image Chip on the left, image with predictions on the right. (Green=TP, Red=FP, Blue=FN)