PixOOD: Pixel-Level Out-of-Distribution Detection

TL;DR

An online data condensation algorithm which is more robust than standard K-means and is easily trainable through SGD, and achieved state-of-the-art results on four out of seven datasets, while being competitive on the rest.

Abstract

We propose a dense image prediction out-of-distribution detection algorithm, called PixOOD, which does not require training on samples of anomalous data and is not designed for a specific application which avoids traditional training biases. In order to model the complex intra-class variability of the in-distribution data at the pixel level, we propose an online data condensation algorithm which is more robust than standard K-means and is easily trainable through SGD. We evaluate PixOOD on a wide range of problems. It achieved state-of-the-art results on four out of seven datasets, while being competitive on the rest. The source code is available at https://github.com/vojirt/PixOOD.

Figures (7)

• Figure 1: Examples of PixOOD results for road, maritime and industrial anomaly detection tasks. PixOOD is able to identify anomalous data not even considered, i.e. not labelled, in standard benchmarks, e.g. power cables (not in Cityscapes training classes) or spilled-out content or scratches in insulation.
• Figure 2: PixOOD inference overview. Individual steps are described in \ref{['sec:method']}. Note that the condensation is run independently for each class during training while MLP is trained jointly for all classes.
• Figure 3: Left: Methods derived in \ref{['sec:condensation']} applied to synthetic data (blue crosses) with outliers (isolated blue crosses). Only the "useful" etalons (from total of 50) with $s_t(k) >= \theta_r$ are displayed as orange circles. (a) K-means is sensitive to outliers and most etalons converge towards isolated data points (not shown) -- only 9 useful etalons. (b) K-medians is more robust -- 13 useful, (c) condensation adds the scale parameter $\beta_k$ to each cluster (black circle) enabling adaptive region of influence -- 15 useful, (d) re-inits with (c) preserve significantly more etalons by combining re-initialization strategy with adaptive scale -- 32 useful. Right: Iterative soft-hard condensation algorithm summary.
• Figure 4: Typical outputs with focus on "failure" cases. PixOOD anomalies are sometimes under-segmented (a) or over-segmented (b,c,m), but it often finds unexpected but reasonable anomalies: a stick (e), reflection in the water (k), birds (l), extra scratches (p), pill remains (q). It is also unable to detect logical anomalies like the switched cable in (n) or missing label in (o). Moreover, semantic/domain shifts are considered as anomaly: mountains (b,c), convertible roof with see-through (d) and city view (f) which are not present in the CityScapes. Legend: red - detected anomaly, green - GT when available, white - ignore region; ellipses mark relevant regions.
• Figure 5: Road anomaly detection examples.