Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explaining raw data complexity to improve satellite onboard processing

Adrien Dorise, Marjorie Bellizzi, Adrien Girard, Benjamin Francesconi, Stéphane May

TL;DR

The paper investigates how processing raw satellite sensor data affects deep learning object detection for onboard processing. It introduces a sensor-aware simulation workflow that converts high-resolution L1 imagery into both simulated L1 and raw-like products, incorporating degradation with $\sigma^2 = \alpha L + \beta$ and a downsampling factor of $x4$, followed by neural restoration and pan-sharpening. Two single-stage detectors, YOLOv11n and YOLOX-S, are trained on L1, simulated L1, and simulated raw data, evaluated with metrics such as mAP and F1, and analyzed with Xplique explanations. Results show modest losses when using simulated L1 versus L1, but raw data degrade high-confidence predictions and boundary localization, underscoring the need for architecture adaptations to robustly process sensor output onboard.

Abstract

With increasing processing power, deploying AI models for remote sensing directly onboard satellites is becoming feasible. However, new constraints arise, mainly when using raw, unprocessed sensor data instead of preprocessed ground-based products. While current solutions primarily rely on preprocessed sensor images, few approaches directly leverage raw data. This study investigates the effects of utilising raw data on deep learning models for object detection and classification tasks. We introduce a simulation workflow to generate raw-like products from high-resolution L1 imagery, enabling systemic evaluation. Two object detection models (YOLOv11n and YOLOX-S) are trained on both raw and L1 datasets, and their performance is compared using standard detection metrics and explainability tools. Results indicate that while both models perform similarly at low to medium confidence thresholds, the model trained on raw data struggles with object boundary identification at high confidence levels. It suggests that adapting AI architectures with improved contouring methods can enhance object detection on raw images, improving onboard AI for remote sensing.

Explaining raw data complexity to improve satellite onboard processing

TL;DR

The paper investigates how processing raw satellite sensor data affects deep learning object detection for onboard processing. It introduces a sensor-aware simulation workflow that converts high-resolution L1 imagery into both simulated L1 and raw-like products, incorporating degradation with and a downsampling factor of , followed by neural restoration and pan-sharpening. Two single-stage detectors, YOLOv11n and YOLOX-S, are trained on L1, simulated L1, and simulated raw data, evaluated with metrics such as mAP and F1, and analyzed with Xplique explanations. Results show modest losses when using simulated L1 versus L1, but raw data degrade high-confidence predictions and boundary localization, underscoring the need for architecture adaptations to robustly process sensor output onboard.

Abstract

With increasing processing power, deploying AI models for remote sensing directly onboard satellites is becoming feasible. However, new constraints arise, mainly when using raw, unprocessed sensor data instead of preprocessed ground-based products. While current solutions primarily rely on preprocessed sensor images, few approaches directly leverage raw data. This study investigates the effects of utilising raw data on deep learning models for object detection and classification tasks. We introduce a simulation workflow to generate raw-like products from high-resolution L1 imagery, enabling systemic evaluation. Two object detection models (YOLOv11n and YOLOX-S) are trained on both raw and L1 datasets, and their performance is compared using standard detection metrics and explainability tools. Results indicate that while both models perform similarly at low to medium confidence thresholds, the model trained on raw data struggles with object boundary identification at high confidence levels. It suggests that adapting AI architectures with improved contouring methods can enhance object detection on raw images, improving onboard AI for remote sensing.

Paper Structure

This paper contains 18 sections, 2 equations, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Materials and methods
  3. Simulation workflow
  4. Degradation
  5. Restoration
  6. Pansharpening
  7. Tiling
  8. Datasets
  9. Models architecture
  10. Experiments
  11. Small objects removal
  12. Model training
  13. Metrics
  14. Results
  15. Validation of the simulation workflow
  16. ...and 3 more sections

Figures (10)

  • Figure 1: Image processing workflow
  • Figure 2: Sensor simulator flowchart
  • Figure 3: Sensor simulation demonstration on a Maxar land product. Maxar Products © 2011-2024 Maxar Technologies.
  • Figure 4: Dataset comparison of the same scene. Maxar Products © 2011-2024 Maxar Technologies.
  • Figure 5: Vessels area and occurence per class
  • ...and 5 more figures