Out-of-Distribution Detection Based on Total Variation Estimation

TL;DR

This work tackles out-of-distribution detection in image classification by introducing TV-OOD, a total-variation-based detector built upon a novel Total Variation Network Estimator (TVNE). By replacing the conventional KL-divergence surrogate (as used in MINE) with total variation, the method yields a score that effectively separates in-distribution from out-of-distribution data and remains robust under mini-batch training. Empirical results across CIFAR-100 and ImageNet-1k, with multiple architectures and with or without an auxiliary OOD dataset ($D_{aug}$), show that TV-OOD matches or outperforms leading baselines on common metrics such as FPR95, AUROC, and AUPR. The findings suggest that total variation is a potent divergence choice for OOD detection in high-dimensional image data and point to promising future theoretical and practical directions for TV-based information-theoretic methods.

Abstract

This paper introduces a novel approach to securing machine learning model deployments against potential distribution shifts in practical applications, the Total Variation Out-of-Distribution (TV-OOD) detection method. Existing methods have produced satisfactory results, but TV-OOD improves upon these by leveraging the Total Variation Network Estimator to calculate each input's contribution to the overall total variation. By defining this as the total variation score, TV-OOD discriminates between in- and out-of-distribution data. The method's efficacy was tested across a range of models and datasets, consistently yielding results in image classification tasks that were either comparable or superior to those achieved by leading-edge out-of-distribution detection techniques across all evaluation metrics.

Figures (2)

• Figure 1: Schematic of TV-OOD implementation: Blue blocks represent the pre-trained classifier $\operatorname{f}$, while red dashed lines/blocks denote the total variation neural estimator $\operatorname{g}(x)$, integrated via an auxiliary dense layer $\operatorname{g_c}(h)$. The left shows training, and the right depicts inference. Both the objective $L$ and OOD score $\operatorname{Sc}$ derive from \ref{['equ:7']}.
• Figure 2: Uniform-sized sub-blocks are extracted from the image. The sub-blocks are sized to be $\frac{1}{64}$ of the original image.