Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond the Visible: Jointly Attending to Spectral and Spatial Dimensions with HSI-Diffusion for the FINCH Spacecraft

Ian Vyse, Rishit Dagli, Dav Vrat Chadha, John P. Ma, Hector Chen, Isha Ruparelia, Prithvi Seran, Matthew Xie, Eesa Aamer, Aidan Armstrong, Naveen Black, Ben Borstein, Kevin Caldwell, Orrin Dahanaggamaarachchi, Joe Dai, Abeer Fatima, Stephanie Lu, Maxime Michet, Anoushka Paul, Carrie Ann Po, Shivesh Prakash, Noa Prosser, Riddhiman Roy, Mirai Shinjo, Iliya Shofman, Coby Silayan, Reid Sox-Harris, Shuhan Zheng, Khang Nguyen

TL;DR

This work addresses the challenge of removing stripe and random noise from hyperspectral images collected by the FINCH CubeSat without distorting spectral information. It introduces a 3D hyperspectral diffusion destriping framework that jointly leverages spatial and spectral cues, trained in three stages on real and synthetically augmented data. Quantitative and qualitative results demonstrate superior destriping performance relative to baselines on synthetic FINCH-like data and EnMAP-like real data, while preserving important scene and spectral characteristics. The approach has practical potential for improving the scientific value of FINCH data and can be extended to broader hyperspectral denoising and super-resolution tasks in remote sensing.

Abstract

Satellite remote sensing missions have gained popularity over the past fifteen years due to their ability to cover large swaths of land at regular intervals, making them ideal for monitoring environmental trends. The FINCH mission, a 3U+ CubeSat equipped with a hyperspectral camera, aims to monitor crop residue cover in agricultural fields. Although hyperspectral imaging captures both spectral and spatial information, it is prone to various types of noise, including random noise, stripe noise, and dead pixels. Effective denoising of these images is crucial for downstream scientific tasks. Traditional methods, including hand-crafted techniques encoding strong priors, learned 2D image denoising methods applied across different hyperspectral bands, or diffusion generative models applied independently on bands, often struggle with varying noise strengths across spectral bands, leading to significant spectral distortion. This paper presents a novel approach to hyperspectral image denoising using latent diffusion models that integrate spatial and spectral information. We particularly do so by building a 3D diffusion model and presenting a 3-stage training approach on real and synthetically crafted datasets. The proposed method preserves image structure while reducing noise. Evaluations on both popular hyperspectral denoising datasets and synthetically crafted datasets for the FINCH mission demonstrate the effectiveness of this approach.

Beyond the Visible: Jointly Attending to Spectral and Spatial Dimensions with HSI-Diffusion for the FINCH Spacecraft

TL;DR

This work addresses the challenge of removing stripe and random noise from hyperspectral images collected by the FINCH CubeSat without distorting spectral information. It introduces a 3D hyperspectral diffusion destriping framework that jointly leverages spatial and spectral cues, trained in three stages on real and synthetically augmented data. Quantitative and qualitative results demonstrate superior destriping performance relative to baselines on synthetic FINCH-like data and EnMAP-like real data, while preserving important scene and spectral characteristics. The approach has practical potential for improving the scientific value of FINCH data and can be extended to broader hyperspectral denoising and super-resolution tasks in remote sensing.

Abstract

Satellite remote sensing missions have gained popularity over the past fifteen years due to their ability to cover large swaths of land at regular intervals, making them ideal for monitoring environmental trends. The FINCH mission, a 3U+ CubeSat equipped with a hyperspectral camera, aims to monitor crop residue cover in agricultural fields. Although hyperspectral imaging captures both spectral and spatial information, it is prone to various types of noise, including random noise, stripe noise, and dead pixels. Effective denoising of these images is crucial for downstream scientific tasks. Traditional methods, including hand-crafted techniques encoding strong priors, learned 2D image denoising methods applied across different hyperspectral bands, or diffusion generative models applied independently on bands, often struggle with varying noise strengths across spectral bands, leading to significant spectral distortion. This paper presents a novel approach to hyperspectral image denoising using latent diffusion models that integrate spatial and spectral information. We particularly do so by building a 3D diffusion model and presenting a 3-stage training approach on real and synthetically crafted datasets. The proposed method preserves image structure while reducing noise. Evaluations on both popular hyperspectral denoising datasets and synthetically crafted datasets for the FINCH mission demonstrate the effectiveness of this approach.
Paper Structure (16 sections, 8 equations, 4 figures, 2 tables)

This paper contains 16 sections, 8 equations, 4 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Contributions.
  3. RELATED WORKS
  4. PRELIMINARY: DIFFUSION MODELS
  5. METHODS
  6. Training the Diffusion Model
  7. Stage I.
  8. Stage II.
  9. Stage III.
  10. Creating Synthetic Data
  11. EXPERIMENTS
  12. Quantitative Results
  13. Qualitative Results
  14. DISCUSSION AND CONCLUSION
  15. Limitations.
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Hyperpsectral Destriping. We present Hyperspectral Diffusion, a technique that can denoise or destripe satellite hyperspectral data cubes. We demonstrate results from real collected data from the EnMAP hyperspectral satellite mission STORCH2023113632rs70708830 which is analogous to images we expect to be captured from our FINCH satellite. Best viewed with color and zoom.
  • Figure 2: Our method trains a 3D diffusion model across 3 stages to perform destriping on hyperspectral satellite images like the ones that would be captured by FINCH.
  • Figure 3: Qualitative Results. We present striped (Row 1) and the corresponding destriped bands produced from our methods (Row 2) from the EnMAP STORCH2023113632rs70708830 image of FINCH's imaging site, with stripes added as described in \ref{['sec:synthetic']}. From left to right, the images correspond to bands 100, 124, 148, 172, 196, and 220. Best viewed with zoom.
  • Figure 4: Qualitative Results. We present striped (Row 1) and the corresponding destriped (Row 2) bands from the EnMAP STORCH2023113632rs70708830 image of FINCH's imaging site, with stripes added as described in \ref{['sec:synthetic']}. From top left to bottom right, the images correspond to bands 100, 124, 148, 172, 196, and 220. We present them in the Viridis viridis colour scheme in this figure, with intensity bars to aid the reader in understanding the image. Best viewed in color and in conjunction with \ref{['fig:enmap-black-and-white']}.