Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty of Network Topology with Applications to Out-of-Distribution Detection

Sing-Yuan Yeh, Chun-Hao Yang

TL;DR

This work introduces predictive topological uncertainty (pTU), a topology-based uncertainty measure for Bayesian neural networks that captures the interaction between inputs and the model via activation-graph persistent diagrams. pTU is defined as the average Fréchet variance of layerwise persistence diagrams across the posterior over weights, linking sample similarity to identical distributions and enabling an architecture-insensitive OOD detector. A theoretical stability result shows Lipschitz continuity of pTU under the Wasserstein distance, and a significance test for covariate shift OOD is proposed based on the Wasserstein distance between pTU distributions, with empirical validation on standard image datasets. Experiments demonstrate pTU’s sensitivity to architecture, effectiveness for OOD detection, and robustness to distributional shifts and data augmentation, highlighting its potential as a principled, topology-informed reliability tool.

Abstract

Persistent homology (PH) is a crucial concept in computational topology, providing a multiscale topological description of a space. It is particularly significant in topological data analysis, which aims to make statistical inference from a topological perspective. In this work, we introduce a new topological summary for Bayesian neural networks, termed the predictive topological uncertainty (pTU). The proposed pTU measures the uncertainty in the interaction between the model and the inputs. It provides insights from the model perspective: if two samples interact with a model in a similar way, then they are considered identically distributed. We also show that the pTU is insensitive to the model architecture. As an application, pTU is used to solve the out-of-distribution (OOD) detection problem, which is critical to ensure model reliability. Failure to detect OOD input can lead to incorrect and unreliable predictions. To address this issue, we propose a significance test for OOD based on the pTU, providing a statistical framework for this issue. The effectiveness of the framework is validated through various experiments, in terms of its statistical power, sensitivity, and robustness.

Uncertainty of Network Topology with Applications to Out-of-Distribution Detection

TL;DR

This work introduces predictive topological uncertainty (pTU), a topology-based uncertainty measure for Bayesian neural networks that captures the interaction between inputs and the model via activation-graph persistent diagrams. pTU is defined as the average Fréchet variance of layerwise persistence diagrams across the posterior over weights, linking sample similarity to identical distributions and enabling an architecture-insensitive OOD detector. A theoretical stability result shows Lipschitz continuity of pTU under the Wasserstein distance, and a significance test for covariate shift OOD is proposed based on the Wasserstein distance between pTU distributions, with empirical validation on standard image datasets. Experiments demonstrate pTU’s sensitivity to architecture, effectiveness for OOD detection, and robustness to distributional shifts and data augmentation, highlighting its potential as a principled, topology-informed reliability tool.

Abstract

Persistent homology (PH) is a crucial concept in computational topology, providing a multiscale topological description of a space. It is particularly significant in topological data analysis, which aims to make statistical inference from a topological perspective. In this work, we introduce a new topological summary for Bayesian neural networks, termed the predictive topological uncertainty (pTU). The proposed pTU measures the uncertainty in the interaction between the model and the inputs. It provides insights from the model perspective: if two samples interact with a model in a similar way, then they are considered identically distributed. We also show that the pTU is insensitive to the model architecture. As an application, pTU is used to solve the out-of-distribution (OOD) detection problem, which is critical to ensure model reliability. Failure to detect OOD input can lead to incorrect and unreliable predictions. To address this issue, we propose a significance test for OOD based on the pTU, providing a statistical framework for this issue. The effectiveness of the framework is validated through various experiments, in terms of its statistical power, sensitivity, and robustness.

Paper Structure

This paper contains 35 sections, 5 theorems, 29 equations, 5 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Bayesian Neural Network
  3. Topological Uncertainty
  4. Contribution
  5. Persistent Homology of Neural Networks
  6. Persistent Homology and Persistence Diagram
  7. Homology
  8. Persistent Homology
  9. Computation
  10. Activation Graph
  11. Topological Uncertainty
  12. Predictive Topological Uncertainty
  13. Model Topological Uncertainty
  14. Stability of pTU under the Wasserstein distance
  15. Significance Test for Out-of-Distribution
  16. ...and 20 more sections

Key Result

Theorem 3.1

Suppose that there are random variables $X \sim P_X$ and $Y \sim P_Y$. Under the assumptions (A.1) -- (A.3), the distribution of $\mathbf{pTU}$ is stable with respect to the $p$-Wasserstein distance for $p \geq 1$, i.e.,

Figures (5)

  • Figure 1: $L$-layer neural network diagram with $L=4$.
  • Figure 2: From left to right, top to bottom, the seven images: the distribution of empirical variance $\{\hat{\sigma}^2_\ell\}_i$ for $\ell=1,\cdots,7$. Each color of line stands for one model. The eighth image: the distribution of $\{\widehat{pTU}(x_i|\mathcal{D}_{\mathtt{F-MNIST}})\}_i$.
  • Figure 3: Power curve for testing OOD. For a model trained by given training dataset $\mathcal{D}_{\texttt{F-MNIST}}^{\bullet}$, each curve stands for the probability of rejecting $H_0$ for $\mu=[0,0.4]$. Blue: the model is trained by non-augmentation dataset $\mathcal{D}_{\texttt{F-MNIST}}$. Orange, green, red, purple: the model is trained by augmentation dataset $\mathcal{D}_{\texttt{F-MNIST}}^{05}$, $\mathcal{D}_{\texttt{F-MNIST}}^{1}$, $\mathcal{D}_{\texttt{F-MNIST}}^{2}$, $\mathcal{D}_{\texttt{F-MNIST}}^{3}$, respectively.
  • Figure 4: Left: the empirical distributions $\{\widehat{pTU}(x_i|\mathcal{D}_{\texttt{SVHN}})\}_{x_i\in\mathcal{X}'}$ where $\mathcal{X}'$ drawn from different underlying distribution. Middle: the empirical distributions $\{\widehat{pTU}(x_i|\mathcal{D}_{\texttt{MNIST}})\}_{x_i\in\mathcal{X}'}$ where $\mathcal{X}'$ drawn from different underlying distribution. Right: the empirical distributions $\{\widehat{pTU}(x_i|\mathcal{D}_{\texttt{F-MNIST}})\}_{x_i\in\mathcal{X}'}$ where $\mathcal{X}'$ drawn from different underlying distribution.
  • Figure 5: Leftmost: five curves show the empirical distributions $\{\widehat{pTU}(s_0(x_i)|\mathcal{D}_{\texttt{F-MNIST}})\}$, $\{\widehat{pTU}(s_0(x_i)|\mathcal{D}_{\texttt{F-MNIST}}^{05})\}$, $\{\widehat{pTU}(s_0(x_i)|\mathcal{D}_{\texttt{F-MNIST}}^{1})\}$, $\{\widehat{pTU}(s_0(x_i)|\mathcal{D}_{\texttt{F-MNIST}}^{2})\}$ and $\{\widehat{pTU}(s_0(x_i)|\mathcal{D}_{\texttt{F-MNIST}}^{3})\}$, respectively. The other subfigures are similar, with the only difference being the shift level, meaning that $s_0$ should be changed to $s_{0.1}$, $s_{0.2}$, $s_{0.3}$ and $s_{0.4}$, respectively.

Theorems & Definitions (9)

  • Definition 3.1: Predictive Topological Uncertainty
  • Definition 3.2: Model Topological Uncertainty
  • Theorem 3.1: Stability of pTU
  • Proposition 6.1
  • Lemma 6.1: Appendix A.2 in lacombe2021topological
  • Lemma 6.2
  • proof
  • Theorem 6.1
  • proof