Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic Segmentation in Satellite Hyperspectral Imagery by Deep Learning

Jon Alvarez Justo, Alexandru Ghita, Daniel Kovac, Joseph L. Garrett, Mariana-Iuliana Georgescu, Jesus Gonzalez-Llorente, Radu Tudor Ionescu, Tor Arne Johansen

TL;DR

A lightweight 1D-CNN model, 1D-Justo-LiuNet, is proposed, which outperforms state-of-the-art models in the hypespectral domain and is highly suitable for in-orbit deployment, providing an effective solution for optimizing and automating satellite operations at edge.

Abstract

Satellites are increasingly adopting on-board AI to optimize operations and increase autonomy through in-orbit inference. The use of Deep Learning (DL) models for segmentation in hyperspectral imagery offers advantages for remote sensing applications. In this work, we train and test 20 models for multi-class segmentation in hyperspectral imagery, selected for their potential in future space deployment. These models include 1D and 2D Convolutional Neural Networks (CNNs) and the latest vision transformers (ViTs). We propose a lightweight 1D-CNN model, 1D-Justo-LiuNet, which outperforms state-of-the-art models in the hypespectral domain. 1D-Justo-LiuNet exceeds the performance of 2D-CNN UNets and outperforms Apple's lightweight vision transformers designed for mobile inference. 1D-Justo-LiuNet achieves the highest accuracy (0.93) with the smallest model size (4,563 parameters) among all tested models, while maintaining fast inference. Unlike 2D-CNNs and ViTs, which encode both spectral and spatial information, 1D-Justo-LiuNet focuses solely on the rich spectral features in hyperspectral data, benefitting from the high-dimensional feature space. Our findings are validated across various satellite datasets, with the HYPSO-1 mission serving as the primary case study for sea, land, and cloud segmentation. We further confirm our conclusions through generalization tests on other hyperspectral missions, such as NASA's EO-1. Based on its superior performance and compact size, we conclude that 1D-Justo-LiuNet is highly suitable for in-orbit deployment, providing an effective solution for optimizing and automating satellite operations at edge.

Semantic Segmentation in Satellite Hyperspectral Imagery by Deep Learning

TL;DR

A lightweight 1D-CNN model, 1D-Justo-LiuNet, is proposed, which outperforms state-of-the-art models in the hypespectral domain and is highly suitable for in-orbit deployment, providing an effective solution for optimizing and automating satellite operations at edge.

Abstract

Satellites are increasingly adopting on-board AI to optimize operations and increase autonomy through in-orbit inference. The use of Deep Learning (DL) models for segmentation in hyperspectral imagery offers advantages for remote sensing applications. In this work, we train and test 20 models for multi-class segmentation in hyperspectral imagery, selected for their potential in future space deployment. These models include 1D and 2D Convolutional Neural Networks (CNNs) and the latest vision transformers (ViTs). We propose a lightweight 1D-CNN model, 1D-Justo-LiuNet, which outperforms state-of-the-art models in the hypespectral domain. 1D-Justo-LiuNet exceeds the performance of 2D-CNN UNets and outperforms Apple's lightweight vision transformers designed for mobile inference. 1D-Justo-LiuNet achieves the highest accuracy (0.93) with the smallest model size (4,563 parameters) among all tested models, while maintaining fast inference. Unlike 2D-CNNs and ViTs, which encode both spectral and spatial information, 1D-Justo-LiuNet focuses solely on the rich spectral features in hyperspectral data, benefitting from the high-dimensional feature space. Our findings are validated across various satellite datasets, with the HYPSO-1 mission serving as the primary case study for sea, land, and cloud segmentation. We further confirm our conclusions through generalization tests on other hyperspectral missions, such as NASA's EO-1. Based on its superior performance and compact size, we conclude that 1D-Justo-LiuNet is highly suitable for in-orbit deployment, providing an effective solution for optimizing and automating satellite operations at edge.
Paper Structure (34 sections, 17 figures, 6 tables)

This paper contains 34 sections, 17 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Methodological Background
  3. Datasets: Labeled Satellite Imagery
  4. Channel Reduction Preprocessing
  5. Semantic Segmentation Modeling
  6. Processing of 1D vs. 3D Data
  7. Baseline Segmentation Models
  8. Customised Semantic Segmentation Models
  9. 2.3.3.1 Review of 1D-CNNs: Liu, Hu, and Lucas CNNs
  10. 2.3.3.2 Ablation Study
  11. 2.3.3.3 Introducing New Lightweigth 1D-CNNs: 1D-Justo-LiuNet, 1D-Justo-HuNet and 1D-Justo-LucasCNN
  12. 2.3.3.4 Introducing a New Lightweight 2D-CNN: 2D-Justo-UNet-Simple
  13. Training Procedure
  14. Segmentation Performance Metrics and Proposed Example Application
  15. Proposed On-Board Sea-Land-Cloud Ranking System
  16. ...and 19 more sections

Figures (17)

  • Figure 1: Lightweight 1D-Justo-LiuNet architecture for on-board segmentation. The block diagram employs 112 input channels for a single pixel. The network generates three detection probabilities, corresponding to sea, land, and clouds, respectively.
  • Figure 2: An automated on-board processing pipeline integrating a ranking system based on sea-land-cloud segmentation. $T\!H_{cl}$, $T\!H_{sea}$, and $T\!H_{land}$ represent thresholds configured at the ground center to regulate data management during in-flight operations.
  • Figure 3: Comparison of false detection ratios for the cloud class by segmentation models trained over 112 channels.
  • Figure 4: Comparison of false detection ratios for the sea class by segmentation models trained over 112 channels.
  • Figure 5: Comparison of false detection ratios for the land class by segmentation models trained over 112 channels.
  • ...and 12 more figures