Deep Learning for Clouds and Cloud Shadow Segmentation in Methane Satellite and Airborne Imaging Spectroscopy

TL;DR

Cloud and cloud-shadow masks are crucial for accurate methane retrievals from hyperspectral data from MethaneSAT and MethaneAIR. The study benchmarks six algorithms, including ILR, MLP, U-Net, SCAN, and two ensemble fusions, demonstrating that combining spatial coherence with spectral attention (Combined CNN) yields the best segmentation performance across both datasets. SCAN excels at boundary delineation, while U-Net provides robust spatial coherence, and their fusion leverages strengths of both, addressing complex scenes and variable surface reflectance. The approach enables on-the-fly masking directly on Level-1 radiance data, improving operational methane quantification and scalability to other hyperspectral platforms. The publicly released annotated datasets and code further support broader adoption and transfer to similar atmospheric sensing tasks.

Abstract

Effective cloud and cloud shadow detection is a critical prerequisite for accurate retrieval of concentrations of atmospheric methane (CH4) or other trace gases in hyperspectral remote sensing. This challenge is especially pertinent for MethaneSAT, a satellite mission launched in March 2024, to fill a significant data gap in terms of resolution, precision and swath between coarse-resolution global mappers and fine-scale point-source imagers of methane, and for its airborne companion mission, MethaneAIR. MethaneSAT delivers hyperspectral data at an intermediate spatial resolution (approx. 100 x 400, m), whereas MethaneAIR provides even finer resolution (approx. 25 m), enabling the development of highly detailed maps of concentrations that enable quantification of both the sources and rates of emissions. In this study, we use machine learning methods to address the cloud and cloud shadow detection problem for sensors with these high spatial resolutions. Cloud and cloud shadows in remote sensing data need to be effectively screened out as they bias methane retrievals in remote sensing imagery and impact the quantification of emissions. We deploy and evaluate conventional techniques-including Iterative Logistic Regression (ILR) and Multilayer Perceptron (MLP)-with advanced deep learning architectures, namely U-Net and a Spectral Channel Attention Network (SCAN) method. Our results show that conventional methods struggle with spatial coherence and boundary definition, affecting the detection of clouds and cloud shadows. Deep learning models substantially improve detection quality: U-Net performs best in preserving spatial structure, while SCAN excels at capturing fine boundary details... Our data and code is publicly available at: https://doi.org/10.7910/DVN/IKLZOJ

Figures (22)

• Figure 1: Hyperspectral methane observations from MethaneSAT and MethaneAIR platforms showing typical mapping coverage and spatial extent. Left: MethaneSAT image at 1640.5 nm wavelength, captured on September 4 of 2024, covering approximately 220 km × 200 km area. Right: MethaneAIR image at 1635.0 nm wavelength, captured on June 2 of 2023, covering approximately 100 km × 100 km area.
• Figure 2: MethaneAIR data example showing classification mask and spectral analysis. The top panel shows a classification mask (purple: background, yellow: clouds, green: shadows, blue: dark surfaces) alongside three images from randomly selected spectral wavelengths. The bottom panel displays normalized radiance spectra from 10 randomly sampled spatial soundings per each class, with colors corresponding to their classification in the mask above. Spectral normalization was performed by computing mean and standard deviation for each spectral band across the entire dataset, then standardizing each spectrum by subtracting the mean and dividing by the standard deviation. This image was captured on September 5, 2023.
• Figure 3: MethaneSAT data example showing classification mask and spectral analysis. The top panel presents the classification mask using the same color scheme as MethaneAIR (purple: background, yellow: clouds, green: shadows) and three representative spectral band images. The bottom panel shows the spectral signatures of 10 randomly selected spatial soundings for each class, with colors indicating their classification status. Spectral normalization was performed by computing mean and standard deviation for each spectral band across the entire dataset, then standardizing each spectrum by subtracting the mean and dividing by the standard deviation. This image was captured on November 18, 2024.
• Figure 4: Methodology overview for cloud and shadow detection in hyperspectral methane monitoring data. The pipeline processes input hyperspectral cubes X$\in \mathbb{R}^{H \times W \times C}$ through preprocessing steps (missing value imputation, spectral normalization) before semantic segmentation. We evaluate six model architectures: spectral-only methods (ILR, MLP), spatial convolution (U-Net), spectral attention (SCAN), and ensemble fusion approaches (Combined MLP/CNN) that integrate predictions from frozen U-Net and SCAN models (dashed arrows). All models are optimized using weighted cross-entropy loss $\ell(f_{\mathcal{W}}(\mathbf{X}), \mathbf{Y})$ to produce pixel-wise cloud and shadow masks Y$\in \{0,1\}^{H \times W \times K}$ with $K=4$ classes (MethaneAIR) or $K=3$ classes (MethaneSAT).
• Figure 5: Radiance at 1592nm, labels and predictions of the Combined CNN model. All figures correspond to the test set from the first cross-validation fold (where the dataset was split into 3 parts, with one part held out for testing)