Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hyperspectral Smoke Segmentation via Mixture of Prototypes

Lujian Yao, Haitao Zhao, Xianghai Kong, Yuhan Xu

TL;DR

The paper tackles smoke segmentation under challenging cloud and semi-transparent conditions by leveraging hyperspectral imaging. It introduces the MoP network, which preserves per-band spectral signatures via band split, models diverse spectral patterns with band-level prototypes, and adaptively weights bands through a dual-level router, achieving superior performance on the HSSDataset and generalizing to MSSDataset. The work provides a first hyperspectral smoke segmentation dataset with a Many-to-One annotation protocol and demonstrates competitive results against strong baselines in both hyperspectral and multispectral settings. This approach offers a path toward more reliable early-warning systems for wildfire management and industrial safety by exploiting rich spectral information.

Abstract

Smoke segmentation is critical for wildfire management and industrial safety applications. Traditional visible-light-based methods face limitations due to insufficient spectral information, particularly struggling with cloud interference and semi-transparent smoke regions. To address these challenges, we introduce hyperspectral imaging for smoke segmentation and present the first hyperspectral smoke segmentation dataset (HSSDataset) with carefully annotated samples collected from over 18,000 frames across 20 real-world scenarios using a Many-to-One annotations protocol. However, different spectral bands exhibit varying discriminative capabilities across spatial regions, necessitating adaptive band weighting strategies. We decompose this into three technical challenges: spectral interaction contamination, limited spectral pattern modeling, and complex weighting router problems. We propose a mixture of prototypes (MoP) network with: (1) Band split for spectral isolation, (2) Prototype-based spectral representation for diverse patterns, and (3) Dual-level router for adaptive spatial-aware band weighting. We further construct a multispectral dataset (MSSDataset) with RGB-infrared images. Extensive experiments validate superior performance across both hyperspectral and multispectral modalities, establishing a new paradigm for spectral-based smoke segmentation.

Hyperspectral Smoke Segmentation via Mixture of Prototypes

TL;DR

The paper tackles smoke segmentation under challenging cloud and semi-transparent conditions by leveraging hyperspectral imaging. It introduces the MoP network, which preserves per-band spectral signatures via band split, models diverse spectral patterns with band-level prototypes, and adaptively weights bands through a dual-level router, achieving superior performance on the HSSDataset and generalizing to MSSDataset. The work provides a first hyperspectral smoke segmentation dataset with a Many-to-One annotation protocol and demonstrates competitive results against strong baselines in both hyperspectral and multispectral settings. This approach offers a path toward more reliable early-warning systems for wildfire management and industrial safety by exploiting rich spectral information.

Abstract

Smoke segmentation is critical for wildfire management and industrial safety applications. Traditional visible-light-based methods face limitations due to insufficient spectral information, particularly struggling with cloud interference and semi-transparent smoke regions. To address these challenges, we introduce hyperspectral imaging for smoke segmentation and present the first hyperspectral smoke segmentation dataset (HSSDataset) with carefully annotated samples collected from over 18,000 frames across 20 real-world scenarios using a Many-to-One annotations protocol. However, different spectral bands exhibit varying discriminative capabilities across spatial regions, necessitating adaptive band weighting strategies. We decompose this into three technical challenges: spectral interaction contamination, limited spectral pattern modeling, and complex weighting router problems. We propose a mixture of prototypes (MoP) network with: (1) Band split for spectral isolation, (2) Prototype-based spectral representation for diverse patterns, and (3) Dual-level router for adaptive spatial-aware band weighting. We further construct a multispectral dataset (MSSDataset) with RGB-infrared images. Extensive experiments validate superior performance across both hyperspectral and multispectral modalities, establishing a new paradigm for spectral-based smoke segmentation.
Paper Structure (21 sections, 10 equations, 14 figures, 9 tables, 2 algorithms)

This paper contains 21 sections, 10 equations, 14 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Smoke Segmentation
  4. Hyperspectral Image
  5. Prototype Learning and Mixture of Experts
  6. Hyperspectral Smoke Segmentation Dataset (HSSDataset)
  7. Hyperspectral Camera System and Data Collection
  8. Dataset Annotation Protocol
  9. Multispectral Smoke Segmentation Dataset (MSSDataset)
  10. Methodology
  11. Overview of Mixture of Prototypes (MoP)
  12. Band Split.
  13. Prototype-based Spectral Representation (Band-level Prototype).
  14. Dual-level Router.
  15. Decoder Head and Loss Function
  16. ...and 6 more sections

Figures (14)

  • Figure 1: Motivation for hyperspectral smoke segmentation. The upper part shows challenging smoke scenarios with cloud interference and semi-transparent regions in the visible light band. The lower part plots the spectral distribution of the marked points. Yellow shaded regions highlight the key discriminatory band ranges where smoke and clouds (or smoke and background) exhibit the most significant spectral divergence.
  • Figure 2: XIMEA MQ022HG-IM-SM5X5-NIR hyperspectral camera specifications and 25-band mosaic filter design. The camera employs a specialized $5 \times 5$ filter array with wavelengths spanning 600-974nm for simultaneous spatial and spectral information.
  • Figure 3: The challenging scenarios of HSSDataset.
  • Figure 4: Many-to-One annotations for hyperspectral smoke segmentation.
  • Figure 5: Visible and infrared frame pairs in MSSDataset.
  • ...and 9 more figures