Margin-bounded Confidence Scores for Out-of-Distribution Detection

TL;DR

A novel and straightforward method called Margin bounded Confidence Scores (MaCS) is proposed to address the nontrivial OOD detection problem by enlarging the disparity between ID and OOD scores, which in turn makes the decision boundary more compact facilitating effective segregation with a simple threshold.

Abstract

In many critical Machine Learning applications, such as autonomous driving and medical image diagnosis, the detection of out-of-distribution (OOD) samples is as crucial as accurately classifying in-distribution (ID) inputs. Recently Outlier Exposure (OE) based methods have shown promising results in detecting OOD inputs via model fine-tuning with auxiliary outlier data. However, most of the previous OE-based approaches emphasize more on synthesizing extra outlier samples or introducing regularization to diversify OOD sample space, which is rather unquantifiable in practice. In this work, we propose a novel and straightforward method called Margin bounded Confidence Scores (MaCS) to address the nontrivial OOD detection problem by enlarging the disparity between ID and OOD scores, which in turn makes the decision boundary more compact facilitating effective segregation with a simple threshold. Specifically, we augment the learning objective of an OE regularized classifier with a supplementary constraint, which penalizes high confidence scores for OOD inputs compared to that of ID and significantly enhances the OOD detection performance while maintaining the ID classification accuracy. Extensive experiments on various benchmark datasets for image classification tasks demonstrate the effectiveness of the proposed method by significantly outperforming state-of-the-art (S.O.T.A) methods on various benchmarking metrics. The code is publicly available at https://github.com/lakpa-tamang9/margin_ood

Figures (5)

• Figure 1: Confidence scores of models trained using CIFAR-100 on test data from CIFAR-100 (ID samples) and iSUN xu2015turkergaze (OOD samples).
• Figure 2: Schematic overview of MaCS where the maximum confidence scores of inputs from $\mathcal{D}_{in}$ and $\mathcal{D}_{out}^{OE}$ are extracted from the output layer of neural network followed by element-wise difference computation between $ID^{max}$ and $OOD^{max}$.
• Figure 3: KDE plot of confidence scores for two OOD test data: iSUN and SVHN against CIFAR-100 ID data trained on a WRN architecture. Left column plots are for MaCS, and right column plots are for OE.
• Figure 4: Line graph representing the OOD detection performance of MaCS across different margin values. Each row represents different ID datasets in the order from top to bottom: CIFAR-10, CIFAR-100, SVHN, Imagenet-32. The results represent an average value over multiple OOD datasets.
• Figure 5: Bar-graph representing different OOD metrics for individual ID datasets with and without margin bound. A WRN model was trained on these ID datasets.