Advancing Out-of-Distribution Detection through Data Purification and Dynamic Activation Function Design

TL;DR

OOD-R (Out-of-Distribution-Rectified), a meticulously curated collection of open-source datasets with enhanced noise reduction properties, is introduced and ActFun, an innovative method that fine-tunes the model's response to diverse inputs, is presented, thereby improving the stability of feature extraction and minimizing specificity issues.

Abstract

In the dynamic realms of machine learning and deep learning, the robustness and reliability of models are paramount, especially in critical real-world applications. A fundamental challenge in this sphere is managing Out-of-Distribution (OOD) samples, significantly increasing the risks of model misclassification and uncertainty. Our work addresses this challenge by enhancing the detection and management of OOD samples in neural networks. We introduce OOD-R (Out-of-Distribution-Rectified), a meticulously curated collection of open-source datasets with enhanced noise reduction properties. In-Distribution (ID) noise in existing OOD datasets can lead to inaccurate evaluation of detection algorithms. Recognizing this, OOD-R incorporates noise filtering technologies to refine the datasets, ensuring a more accurate and reliable evaluation of OOD detection algorithms. This approach not only improves the overall quality of data but also aids in better distinguishing between OOD and ID samples, resulting in up to a 2.5\% improvement in model accuracy and a minimum 3.2\% reduction in false positives. Furthermore, we present ActFun, an innovative method that fine-tunes the model's response to diverse inputs, thereby improving the stability of feature extraction and minimizing specificity issues. ActFun addresses the common problem of model overconfidence in OOD detection by strategically reducing the influence of hidden units, which enhances the model's capability to estimate OOD uncertainty more accurately. Implementing ActFun in the OOD-R dataset has led to significant performance enhancements, including an 18.42\% increase in AUROC of the GradNorm method and a 16.93\% decrease in FPR95 of the Energy method. Overall, our research not only advances the methodologies in OOD detection but also emphasizes the importance of dataset integrity for accurate algorithm evaluation.

Figures (4)

• Figure 1: The illustration of the out-of-distribution (OOD) detection process adopted by our classifier. Texturecimpoi2014describing, Places365zhou2017places, SUN subsetxiao2010sun and other datasets are taken as input. The classifier predicts the input data through the network category. The rightmost part of the figure highlights the OOD detection mechanism. It shows that some samples (such as the two pictures shown with orange borders) are correctly classified and, therefore, considered IDs, but some samples (such as the blue picture shown in the border) are accurately identified as OOD. This shows that some samples of classification pairs are mistakenly placed in the OOD test set.
• Figure 2: Categorization of data samples within various OOD datasets. This figure demonstrates the classification results where each image is labeled as either ID or OOD across different datasets. Red boxes indicate images incorrectly labeled as ID within OOD datasets (false negatives), and green boxes signify correctly identified OOD samples (true positives). The results highlight the challenge of distinguishing complex patterns in OOD detection tasks and the importance of accurate labeling for the optimization of OOD algorithms.
• Figure 3: Performance of various OOD detection algorithms across different datasets, before and after noise reduction. Each subplot represents a different detection method, with the solid lines indicating the detection performance on the original OOD datasets and the dashed lines showing performance on the noise-reduced datasets (OOD-R). The blue and red lines correspond to the BiTkolesnikov2020big and VGGding2021repvgg models, respectively. This analysis demonstrates the effects of noise reduction on the sensitivity and specificity of OOD detection methods, with varying degrees of impact observed across different methods and datasets.
• Figure 4: Comparative performance of OOD detection methods with varying $\beta$ values for the Softplus activation function. ViMwang2022vim, KL-Matchinghendrycks2019scaling, Residual and Mahalanobislee2018simple demonstrate how an increase in beta affects assay sensitivity and specificity. The ViMwang2022vim and KL-Matchinghendrycks2019scaling methods show improved or stable detection rates as $\beta$ increases, whereas the Residual and Mahalanobislee2018simple methods exhibit increased false positives, indicating a sensitivity to the activation function's smoothness. The results highlight the critical role of $\beta$ in balancing model sensitivity and robustness for OOD detection.