Neural Network Meta Classifier: Improving the Reliability of Anomaly Segmentation

TL;DR

This work tackles anomaly segmentation in open-set semantic segmentation by leveraging entropy maximization to produce per-pixel anomaly scores. It replaces the traditional logistic-regression meta classifier with a lightweight neural network, and introduces informative out-of-distribution examples to improve training, while revealing that the NN’s behavior remains closely related to the LR baseline. Empirical results show the neural network meta classifier achieves higher AUROC and AUPRC (e.g., AUROC $0.9680$ vs $0.9440$, AUPRC $0.8418$ vs $0.6819$) on challenging OoD cases, especially for small anomalies. The paper also provides guidance on the value of informative OoD proxies and discusses interpretability considerations, making the approach practical for safer open-set semantic segmentation in real-world systems.

Abstract

Deep neural networks (DNNs) are a contemporary solution for semantic segmentation and are usually trained to operate on a predefined closed set of classes. In open-set environments, it is possible to encounter semantically unknown objects or anomalies. Road driving is an example of such an environment in which, from a safety standpoint, it is important to ensure that a DNN indicates it is operating outside of its learned semantic domain. One possible approach to anomaly segmentation is entropy maximization, which is paired with a logistic regression based post-processing step called meta classification, which is in turn used to improve the reliability of detection of anomalous pixels. We propose to substitute the logistic regression meta classifier with a more expressive lightweight fully connected neural network. We analyze advantages and drawbacks of the proposed neural network meta classifier and demonstrate its better performance over logistic regression. We also introduce the concept of informative out-of-distribution examples which we show to improve training results when using entropy maximization in practice. Finally, we discuss the loss of interpretability and show that the behavior of logistic regression and neural network is strongly correlated.

Figures (4)

• Figure 1: ROC and PR meta classifier curves for OoD object predictions of LostAndFound Test images. On the PR curve, random guessing is represented as a constant dashed red line whose value is equal to the ratio of the number of OoD objects and the total number of predicted OoD objects.
• Figure 2: Examples of high and low informative proxy OoD images. The first row contains the proxy OoD images while the second row contains ground truth segmentation masks such that the white regions represent pixels labeled as OoD for which Eq. (\ref{['eq:l_out_part_of_the_objective']}) is applied.
• Figure 3: LARS path for the hand-crafted metrics at $t = 0.7$. A detailed description of the hand-crafted metrics can be found in DBLP:journals/corr/abs-2012-06575.
• Figure 4: Performance comparison of logistic regression meta classifier and neural network meta classifier when trained on subsets of the hand-crafted metrics dataset $\mu$. For each value $N_{m}$ on the x-axis, we train the meta classifiers on the subset of $\mu$ such that we take the first $N_{m}$ metrics having the most correlation with the response according to LARS.