Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatial-Aware Conformal Prediction for Trustworthy Hyperspectral Image Classification

Kangdao Liu, Tianhao Sun, Hao Zeng, Yongshan Zhang, Chi-Man Pun, Chi-Man Vong

TL;DR

A rigorous theoretical proof is presented, which demonstrates the validity of Conformal Prediction, an emerging uncertainty quantification technique, in the context of HSI classification, and a novel framework of Conformal Prediction specifically designed for HSI data, called Spatial-Aware Conformal Prediction ( SACP).

Abstract

Hyperspectral image (HSI) classification involves assigning unique labels to each pixel to identify various land cover categories. While deep classifiers have achieved high predictive accuracy in this field, they lack the ability to rigorously quantify confidence in their predictions. Quantifying the certainty of model predictions is crucial for the safe usage of predictive models, and this limitation restricts their application in critical contexts where the cost of prediction errors is significant. To support the safe deployment of HSI classifiers, we first provide a theoretical proof establishing the validity of the emerging uncertainty quantification technique, conformal prediction, in the context of HSI classification. We then propose a conformal procedure that equips any trained HSI classifier with trustworthy prediction sets, ensuring that these sets include the true labels with a user-specified probability (e.g., 95\%). Building on this foundation, we introduce Spatial-Aware Conformal Prediction (\texttt{SACP}), a conformal prediction framework specifically designed for HSI data. This method integrates essential spatial information inherent in HSIs by aggregating the non-conformity scores of pixels with high spatial correlation, which effectively enhances the efficiency of prediction sets. Both theoretical and empirical results validate the effectiveness of our proposed approach. The source code is available at \url{https://github.com/J4ckLiu/SACP}.

Spatial-Aware Conformal Prediction for Trustworthy Hyperspectral Image Classification

TL;DR

A rigorous theoretical proof is presented, which demonstrates the validity of Conformal Prediction, an emerging uncertainty quantification technique, in the context of HSI classification, and a novel framework of Conformal Prediction specifically designed for HSI data, called Spatial-Aware Conformal Prediction ( SACP).

Abstract

Hyperspectral image (HSI) classification involves assigning unique labels to each pixel to identify various land cover categories. While deep classifiers have achieved high predictive accuracy in this field, they lack the ability to rigorously quantify confidence in their predictions. Quantifying the certainty of model predictions is crucial for the safe usage of predictive models, and this limitation restricts their application in critical contexts where the cost of prediction errors is significant. To support the safe deployment of HSI classifiers, we first provide a theoretical proof establishing the validity of the emerging uncertainty quantification technique, conformal prediction, in the context of HSI classification. We then propose a conformal procedure that equips any trained HSI classifier with trustworthy prediction sets, ensuring that these sets include the true labels with a user-specified probability (e.g., 95\%). Building on this foundation, we introduce Spatial-Aware Conformal Prediction (\texttt{SACP}), a conformal prediction framework specifically designed for HSI data. This method integrates essential spatial information inherent in HSIs by aggregating the non-conformity scores of pixels with high spatial correlation, which effectively enhances the efficiency of prediction sets. Both theoretical and empirical results validate the effectiveness of our proposed approach. The source code is available at \url{https://github.com/J4ckLiu/SACP}.
Paper Structure (34 sections, 4 theorems, 42 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 34 sections, 4 theorems, 42 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Conformal Prediction
  4. Notation
  5. Pipeline
  6. Non-conformity score function
  7. Evaluation Metric
  8. HSI Classification
  9. Validity of Conformal Prediction on HSI
  10. Theoretical Analysis
  11. Conformal HSI Classification
  12. Spatial-Aware Conformal Prediction
  13. Motivation
  14. Methodology
  15. Experiments
  16. ...and 19 more sections

Key Result

Theorem 1

vovk2005algorithmic Let $(\bm{x}_i, y_i)_{i=1}^{n}$ and $(\boldsymbol{x}_{n+1}, y_{n+1})$ be exchangeable data samples. Suppose $\tau$ is calculated by Eq. eq:calculate_tau using $(\bm{x}_i, y_i)_{i=1}^{n}$, and the prediction set for $(\boldsymbol{x}_{n+1}, y_{n+1})$ is given by Eq. equ4. Then, the

Figures (4)

  • Figure 1: Overview of SACP: Raw non-conformity scores lead to excessively large prediction sets. SACP addresses this issue by refining the scores through aggregation of instances with high spatial correlation. This approach is a simple yet effective solution for enhancing the efficiency of prediction sets.
  • Figure 2: Visualizations of the average size of prediction sets for each pixel in $\mathcal{D}_{\text{cal}}$ and $\mathcal{D}_{\text{test}}$ are presented for the (a) IP and (e) PU datasets. The true labels for each class are shown in (b) and (f). The brightness of each pixel indicates the size of its prediction set, with brighter pixels representing larger sets. Panels (c) and (g) display the set sizes with SACP, while panels (d) and (h) show the set sizes with standard conformal prediction.
  • Figure 3: Effect of $\lambda$ and $k$ on Prediction Set Size: Each subplot presents results from three comparative experiments conducted on different datasets. The dashed line indicates the performance of APS without SACP.
  • Figure 4: Size vs. Difficulty: The term "Difficulty" refers to the ranking of the true class label in the output probability, reflecting the difficulty of accurately classifying the pixel. We report the average size of prediction sets for the PU and SA datasets using SSTN, categorized by their Difficulty.

Theorems & Definitions (7)

  • Theorem 1
  • Theorem 2
  • Proposition 1
  • Proposition 2
  • proof
  • proof
  • proof