Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing

Alaa Anani, Tobias Lorenz, Bernt Schiele, Mario Fritz

TL;DR

The paper tackles high abstain rates in segmentation certification by introducing an adaptive hierarchical framework that certifies pixels within a multi-level semantic hierarchy using randomized smoothing. It formalizes a hierarchical smoothed model $c^{\tau,H}$ and a practical AdaptiveCertify algorithm that maps fluctuating components to coarser labels while preserving probabilistic guarantees, controlled by a threshold $\tau$ and radius $R = \sigma \Phi^{-1}(\tau)$. A novel Certified Information Gain (CIG) metric quantifies the information preserved by different hierarchy levels and is validated on Cityscapes, ACDC, PASCAL-Context, and COCO-Stuff, showing higher CIG and lower abstention than the state-of-the-art SegCertify, especially in datasets with many fine-grained classes. The approach combines a class hierarchy graph, adaptive sampling, and multiple hypothesis testing to provide semantically meaningful, certifiable outputs for safety-critical segmentation tasks, with a framework that generalizes to black-box models and arbitrary hierarchies.

Abstract

Certification for machine learning is proving that no adversarial sample can evade a model within a range under certain conditions, a necessity for safety-critical domains. Common certification methods for segmentation use a flat set of fine-grained classes, leading to high abstain rates due to model uncertainty across many classes. We propose a novel, more practical setting, which certifies pixels within a multi-level hierarchy, and adaptively relaxes the certification to a coarser level for unstable components classic methods would abstain from, effectively lowering the abstain rate whilst providing more certified semantically meaningful information. We mathematically formulate the problem setup, introduce an adaptive hierarchical certification algorithm and prove the correctness of its guarantees. Since certified accuracy does not take the loss of information into account for coarser classes, we introduce the Certified Information Gain ($\mathrm{CIG}$) metric, which is proportional to the class granularity level. Our extensive experiments on the datasets Cityscapes, PASCAL-Context, ACDC and COCO-Stuff demonstrate that our adaptive algorithm achieves a higher $\mathrm{CIG}$ and lower abstain rate compared to the current state-of-the-art certification method. Our code can be found here: https://github.com/AlaaAnani/adaptive-certify.

Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing

TL;DR

The paper tackles high abstain rates in segmentation certification by introducing an adaptive hierarchical framework that certifies pixels within a multi-level semantic hierarchy using randomized smoothing. It formalizes a hierarchical smoothed model and a practical AdaptiveCertify algorithm that maps fluctuating components to coarser labels while preserving probabilistic guarantees, controlled by a threshold and radius . A novel Certified Information Gain (CIG) metric quantifies the information preserved by different hierarchy levels and is validated on Cityscapes, ACDC, PASCAL-Context, and COCO-Stuff, showing higher CIG and lower abstention than the state-of-the-art SegCertify, especially in datasets with many fine-grained classes. The approach combines a class hierarchy graph, adaptive sampling, and multiple hypothesis testing to provide semantically meaningful, certifiable outputs for safety-critical segmentation tasks, with a framework that generalizes to black-box models and arbitrary hierarchies.

Abstract

Certification for machine learning is proving that no adversarial sample can evade a model within a range under certain conditions, a necessity for safety-critical domains. Common certification methods for segmentation use a flat set of fine-grained classes, leading to high abstain rates due to model uncertainty across many classes. We propose a novel, more practical setting, which certifies pixels within a multi-level hierarchy, and adaptively relaxes the certification to a coarser level for unstable components classic methods would abstain from, effectively lowering the abstain rate whilst providing more certified semantically meaningful information. We mathematically formulate the problem setup, introduce an adaptive hierarchical certification algorithm and prove the correctness of its guarantees. Since certified accuracy does not take the loss of information into account for coarser classes, we introduce the Certified Information Gain () metric, which is proportional to the class granularity level. Our extensive experiments on the datasets Cityscapes, PASCAL-Context, ACDC and COCO-Stuff demonstrate that our adaptive algorithm achieves a higher and lower abstain rate compared to the current state-of-the-art certification method. Our code can be found here: https://github.com/AlaaAnani/adaptive-certify.
Paper Structure (34 sections, 2 theorems, 9 equations, 17 figures, 7 tables, 5 algorithms)

This paper contains 34 sections, 2 theorems, 9 equations, 17 figures, 7 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries: Randomized Smoothing for Segmentation
  4. Segmentation.
  5. Adaptive Hierarchical Certification
  6. Hierarchical Certification: Formulation
  7. The Class Hierarchy Graph
  8. Fluctuating Components and Adaptive Sampling
  9. Our Algorithm: AdaptiveCertify
  10. Properties of AdaptiveCertify
  11. Evaluation Paradigm: Certified Information Gain
  12. Results
  13. Conclusion
  14. Algorithm functions
  15. Experimental setup
  16. ...and 19 more sections

Key Result

Theorem 3.1

Let $\mathcal{I}_x=\{i \mid g_i^{\tau} \neq \oslash, i\in 1, \texttt{...}, N\}$ be the set of certified components indices in $x$. Then, for a perturbation $\delta \in \mathbb{R}^{N\times m}$ with $||\delta||_2 < R \coloneqq \sigma \Phi^{-1}(\tau)$, for all $i\in \mathcal{I}_x$: $g_i^{\tau}(x+\delta

Figures (17)

  • Figure 1: The certified segmentation outputs on input images (a) and (d) from SegCertify in (b) and (e), and AdaptiveCertify in (c) and (f) with their corresponding Certified Information Gain ($\mathrm{CIG}$) and abstain rate %$\oslash$. Our method provides more meaningful certified output in pixels the state-of-the-art abstains from (white pixels), with a much lower abstain rate, and higher $\mathrm{CIG}$. The segmentation color palette can be found in Figure \ref{['fig:cs-h']}.
  • Figure 2: A DAG representing a semantic hierarchy on top of the $19$ classes of Cityscapes cityscapes. The node colors represent the color palette used in the segmentation results. Hierarchies on all datasets are described in App. \ref{['subsection:datasets']}.
  • Figure 3: The performance of AdaptiveCertify against the baseline in terms of the difference in $\mathrm{CIG}$ ($\Delta \mathrm{CIG}$) and the certification rate ($\Delta \%$certified) across the 3 datasets w.r.t their top classes. "Overall" indicates the class-average performance. Extensions of this figure to Cityscapes and all classes in PASCAL-Context and COCO-Stuff-10K datasets are in App. Figures \ref{['fig:delta-cs']}, \ref{['fig:pascal-delta-ext']} and \ref{['fig:coco-delta-ext']} under App. \ref{['subsection:delta']}.
  • Figure 4: The performance of our algorithm against the baseline in terms of percentage of abstain and certified pixels under different hierarchy levels. SegCertify, by definition, only uses $H_0$.
  • Figure 5: The frequency of the top-most sets of classes the model fluctuates between in abstain pixels by the baseline on the COCO-Stuff-10K and PASCAL-Context datasets.
  • ...and 12 more figures

Theorems & Definitions (3)

  • Theorem 3.1: from fischer
  • Proposition 1: Similar to fischer
  • proof