Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty and Prediction Quality Estimation for Semantic Segmentation via Graph Neural Networks

Edgar Heinert, Stephan Tilgner, Timo Palm, Matthias Rottmann

Abstract

When employing deep neural networks (DNNs) for semantic segmentation in safety-critical applications like automotive perception or medical imaging, it is important to estimate their performance at runtime, e.g. via uncertainty estimates or prediction quality estimates. Previous works mostly performed uncertainty estimation on pixel-level. In a line of research, a connected-component-wise (segment-wise) perspective was taken, approaching uncertainty estimation on an object-level by performing so-called meta classification and regression to estimate uncertainty and prediction quality, respectively. In those works, each predicted segment is considered individually to estimate its uncertainty or prediction quality. However, the neighboring segments may provide additional hints on whether a given predicted segment is of high quality, which we study in the present work. On the basis of uncertainty indicating metrics on segment-level, we use graph neural networks (GNNs) to model the relationship of a given segment's quality as a function of the given segment's metrics as well as those of its neighboring segments. We compare different GNN architectures and achieve a notable performance improvement.

Uncertainty and Prediction Quality Estimation for Semantic Segmentation via Graph Neural Networks

Abstract

When employing deep neural networks (DNNs) for semantic segmentation in safety-critical applications like automotive perception or medical imaging, it is important to estimate their performance at runtime, e.g. via uncertainty estimates or prediction quality estimates. Previous works mostly performed uncertainty estimation on pixel-level. In a line of research, a connected-component-wise (segment-wise) perspective was taken, approaching uncertainty estimation on an object-level by performing so-called meta classification and regression to estimate uncertainty and prediction quality, respectively. In those works, each predicted segment is considered individually to estimate its uncertainty or prediction quality. However, the neighboring segments may provide additional hints on whether a given predicted segment is of high quality, which we study in the present work. On the basis of uncertainty indicating metrics on segment-level, we use graph neural networks (GNNs) to model the relationship of a given segment's quality as a function of the given segment's metrics as well as those of its neighboring segments. We compare different GNN architectures and achieve a notable performance improvement.
Paper Structure (5 sections, 5 equations, 3 figures, 3 tables)

This paper contains 5 sections, 5 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neighbor-Aware Meta Classification and Regression
  4. Numerical Experiments
  5. Conclusion and Outlook

Figures (3)

  • Figure 1: An illustration of our graph-based uncertainty estimation and prediction quality estimation approach.
  • Figure 2: Cityscapes image overlayed with a graph generated from segment barycenters. Each segment is connected to adjacent ones
  • Figure 3: Illustration of the characteristics of the Bayesian optimization results for the top 80th ranked model architectures. (a) AUROC course over the model rank (models ordered from highest to lowest AUROC). (b) histogram of the number of layers in the model (orange) and the number of GNN-based layers within the architecture (blue). (c) distribution of the learning rate of the model architectures. (d) histogram of the number of neurons in the layers (blue, orange, green), except each last layer with only one fixed neuron, and the total number of neurons in the architecture (red).