Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Parallel Workflow for Polar Sea-Ice Classification using Auto-labeling of Sentinel-2 Imagery

Jurdana Masuma Iqrah, Wei Wang, Hongjie Xie, Sushil Prasad

TL;DR

A scalable and accurate method for segmenting and automatically labeling S2 images using carefully determined color thresholds is demonstrated and the U-Net model trained on auto-labeled data achieves a classification accuracy of 98.97% for auto-labeled training datasets when the thin clouds and shadows from the S2 images are filtered out.

Abstract

The observation of the advancing and retreating pattern of polar sea ice cover stands as a vital indicator of global warming. This research aims to develop a robust, effective, and scalable system for classifying polar sea ice as thick/snow-covered, young/thin, or open water using Sentinel-2 (S2) images. Since the S2 satellite is actively capturing high-resolution imagery over the earth's surface, there are lots of images that need to be classified. One major obstacle is the absence of labeled S2 training data (images) to act as the ground truth. We demonstrate a scalable and accurate method for segmenting and automatically labeling S2 images using carefully determined color thresholds. We employ a parallel workflow using PySpark to scale and achieve 9-fold data loading and 16-fold map-reduce speedup on auto-labeling S2 images based on thin cloud and shadow-filtered color-based segmentation to generate label data. The auto-labeled data generated from this process are then employed to train a U-Net machine learning model, resulting in good classification accuracy. As training the U-Net classification model is computationally heavy and time-consuming, we distribute the U-Net model training to scale it over 8 GPUs using the Horovod framework over a DGX cluster with a 7.21x speedup without affecting the accuracy of the model. Using the Antarctic's Ross Sea region as an example, the U-Net model trained on auto-labeled data achieves a classification accuracy of 98.97% for auto-labeled training datasets when the thin clouds and shadows from the S2 images are filtered out.

A Parallel Workflow for Polar Sea-Ice Classification using Auto-labeling of Sentinel-2 Imagery

TL;DR

A scalable and accurate method for segmenting and automatically labeling S2 images using carefully determined color thresholds is demonstrated and the U-Net model trained on auto-labeled data achieves a classification accuracy of 98.97% for auto-labeled training datasets when the thin clouds and shadows from the S2 images are filtered out.

Abstract

The observation of the advancing and retreating pattern of polar sea ice cover stands as a vital indicator of global warming. This research aims to develop a robust, effective, and scalable system for classifying polar sea ice as thick/snow-covered, young/thin, or open water using Sentinel-2 (S2) images. Since the S2 satellite is actively capturing high-resolution imagery over the earth's surface, there are lots of images that need to be classified. One major obstacle is the absence of labeled S2 training data (images) to act as the ground truth. We demonstrate a scalable and accurate method for segmenting and automatically labeling S2 images using carefully determined color thresholds. We employ a parallel workflow using PySpark to scale and achieve 9-fold data loading and 16-fold map-reduce speedup on auto-labeling S2 images based on thin cloud and shadow-filtered color-based segmentation to generate label data. The auto-labeled data generated from this process are then employed to train a U-Net machine learning model, resulting in good classification accuracy. As training the U-Net classification model is computationally heavy and time-consuming, we distribute the U-Net model training to scale it over 8 GPUs using the Horovod framework over a DGX cluster with a 7.21x speedup without affecting the accuracy of the model. Using the Antarctic's Ross Sea region as an example, the U-Net model trained on auto-labeled data achieves a classification accuracy of 98.97% for auto-labeled training datasets when the thin clouds and shadows from the S2 images are filtered out.
Paper Structure (22 sections, 14 figures, 5 tables)

This paper contains 22 sections, 14 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Sentinel-2 Data Collection and Preprocessing
  5. Auto-labeling Method
  6. Python Multiprocessing
  7. PySpark Map-Reduce
  8. U-Net Model for Sea-Ice Classification
  9. U-Net Training
  10. U-Net Model Inferencing
  11. Experiments and Results
  12. Experimental Setup and Evaluation Metrics
  13. Auto-labeling
  14. Auto-labeling Speedup
  15. Python Multiprocessing Performance
  16. ...and 7 more sections

Figures (14)

  • Figure 1: Workflow for parallel and distributed auto-labeling and U-Net model training.
  • Figure 2: Workflow for training and test data preparation and sea ice classification methodology.
  • Figure 3: Sample Sentinel-2 scenes, (a) with thin cloud/shadow cover, and (b) without cloud or shadows.
  • Figure 4: Manually-labeled data and their color codes, (a), (b), and (c) represent original Sentinel-2 data, (d), (e), and (f) represent the respective manually-labeled data.
  • Figure 5: Thin Cloud and Shadow Filtered Dataset, (a), (b) and (c) represent Sentinel-2 thin cloudy and shadowy images and (d), (e) and (f) as the corresponding filtered images.
  • ...and 9 more figures