Effects of Common Regularization Techniques on Open-Set Recognition

TL;DR

It is shown empirically that regularization methods can provide significant improvements to Open-Set Recognition performance, and new insights are provided into the relationship between accuracy and Open-Set performance.

Abstract

In recent years there has been increasing interest in the field of Open-Set Recognition, which allows a classification model to identify inputs as "unknown" when it encounters an object or class not in the training set. This ability to flag unknown inputs is of vital importance to many real world classification applications. As almost all modern training methods for neural networks use extensive amounts of regularization for generalization, it is therefore important to examine how regularization techniques impact the ability of a model to perform Open-Set Recognition. In this work, we examine the relationship between common regularization techniques and Open-Set Recognition performance. Our experiments are agnostic to the specific open-set detection algorithm and examine the effects across a wide range of datasets. We show empirically that regularization methods can provide significant improvements to Open-Set Recognition performance, and we provide new insights into the relationship between accuracy and Open-Set performance.

Figures (5)

• Figure 1: Example of Mixup and CutMix on Tiny ImageNet data.
• Figure 2: Example of desirable and undesirable histogram overlap.
• Figure 3: Closed-set accuracy vs AUROC for the CI 10+ dataset across three runs.
• Figure 4: Closed-set accuracy vs AUROC for TIN 100-100 across three runs.
• Figure 5: Example of visually similar FGVCA classes.