Polyhedral Unmixing: Bridging Semantic Segmentation with Hyperspectral Unmixing via Polyhedral-Cone Partitioning

Abstract

Semantic segmentation and hyperspectral unmixing are two central problems in spectral image analysis. The former assigns each pixel a discrete label corresponding to its material class, whereas the latter estimates pure material spectra, called endmembers, and, for each pixel, a vector representing material abundances in the observed scene. Despite their complementarity, these two problems are usually addressed independently. This paper aims to bridge these two lines of work by formally showing that, under the linear mixing model, pixel classification by dominant materials induces polyhedral-cone regions in the spectral space. We leverage this fundamental property to propose a direct segmentation-to-unmixing pipeline that performs blind hyperspectral unmixing from any semantic segmentation by constructing a polyhedral-cone partition of the space that best fits the labeled pixels. Signed distances from pixels to the estimated regions are then computed, linearly transformed via a change of basis in the distance space, and projected onto the probability simplex, yielding an initial abundance estimate. This estimate is used to extract endmembers and recover final abundances via matrix pseudo-inversion. Because the segmentation method can be freely chosen, the user gains explicit control over the unmixing process, while the rest of the pipeline remains essentially deterministic and lightweight. Beyond improving interpretability, experiments on three real datasets demonstrate the effectiveness of the proposed approach when associated with appropriate clustering algorithms, and show consistent improvements over recent deep and non-deep state-of-the-art methods. The code is available at: https://github.com/antoine-bottenmuller/polyhedral-unmixing

Figures (12)

• Figure 1: Bridge between semantic segmentation and hyperspectral unmixing. From the unmixing data, the dominant-material classification map is directly obtained by taking the $\arg\max$ over the abundances for each pixel. In the opposite way, a model is needed.
• Figure 2: Illustration, in the spectral space, of the standard pipeline for blind linear umixing. 2D cross-sections of a 3D space.
• Figure 3: Processing chain to determine a polyhedral-cone partition of the spectral space. From left to right: observed data (A.) are pre-processed (B.) and classified into $m$ classes (C.). Separation hyperplanes are then computed (D.), inducing $k \geq m$ polyhedral regions. Top row: image domain; bottom row: spectral space representation (3D view + 2D projection plane $P(\vec{x}, \vec{y})$ below, which allows a better visualization of the data and the three polyhedral regions). Each RGB color is associated with one class.
• Figure 4: Processing chain for computing an abundance estimate. From left to right: given the spectral partition (D.), signed distances to each cone are computed (E.); a change of basis is then applied in the distance space (F.), and the resulting vectors are projected onto the probability simplex (G.). Top: image; middle: spectral space; bottom: distance space. The 2D RGB projection plane $P(\vec{x}, \vec{y})$ shows: polyhedral regions (D.), negative distances in each region (E.), transformed distance vectors (F.) and abundance vectors (G.). Each RGB color is associated with one class. Yellow, magenta and cyan represent separation planes between R-G, R-B and G-B pairs of classes, respectively.
• Figure 5: The Samson dataset (\ref{['subfig:samson_y']}) with its ground-truth (GT) abundance map (\ref{['subfig:samson_a']}) and endmembers (\ref{['subfig:samson_m']}). In subfigures (\ref{['subfig:samson_a']}) and (\ref{['subfig:samson_m']}), red is associated with the soil class, green with trees, and blue with water.

Theorems & Definitions (10)

• Definition 1: Halfspace
• Definition 2: Polyhedron
• Definition 3: Polyhedral Cone
• Definition 4: Dominant-Material Region
• Lemma 1: Region convexity
• Lemma 2: Polyhedral Property of Convex Partitions
• Theorem 1: Polyhedral-Cone Partition Theorem
• proof
• proof
• proof