Conservative Prediction via Data-Driven Confidence Minimization

TL;DR

DCM introduces a confidence-minimizing regularizer over an uncertainty dataset to train models that abstain or defer when inputs are uncertain. By combining standard cross-entropy on labeled data with a uniform-target confidence loss on unlabeled uncertain inputs, DCM provably separates unknown test examples from known ones under mild assumptions. It unifies selective classification and OOD detection within a single framework and demonstrates consistent improvements over state-of-the-art methods across multiple benchmarks, including distribution shifts and large-scale datasets. The approach offers practical benefits for safety-critical deployments, though it requires careful construction of the uncertainty data and distribution-aware fine-tuning.

Abstract

In safety-critical applications of machine learning, it is often desirable for a model to be conservative, abstaining from making predictions on unknown inputs which are not well-represented in the training data. However, detecting unknown examples is challenging, as it is impossible to anticipate all potential inputs at test time. To address this, prior work (Hendrycks et al., 2018) minimizes model confidence on an auxiliary outlier dataset carefully curated to be disjoint from the training distribution. We theoretically analyze the choice of auxiliary dataset for confidence minimization, revealing two actionable insights: (1) if the auxiliary set contains unknown examples similar to those seen at test time, confidence minimization leads to provable detection of unknown test examples, and (2) if the first condition is satisfied, it is unnecessary to filter out known examples for out-of-distribution (OOD) detection. Motivated by these guidelines, we propose the Data-Driven Confidence Minimization (DCM) framework, which minimizes confidence on an uncertainty dataset. We apply DCM to two problem settings in which conservative prediction is paramount -- selective classification and OOD detection -- and provide a realistic way to gather uncertainty data for each setting. In our experiments, DCM consistently outperforms existing selective classification approaches on 4 datasets when tested on unseen distributions and outperforms state-of-the-art OOD detection methods on 12 ID-OOD dataset pairs, reducing FPR (at TPR $95\%$) by $6.3\%$ and $58.1\%$ on CIFAR-10 and CIFAR-100 compared to Outlier Exposure.

Figures (7)

• Figure 1: Data-driven confidence minimization (DCM) is a framework for training a model to make conservative predictions. DCM incorporates a regularizer that minimizes confidence on an unlabeled mixture of known and unknown examples that are similar to those seen at test-time.
• Figure 2: Selective classification performance of DCM on the CIFAR-10 $\rightarrow$ CIFAR-10-C task with validation set sizes (left) and various confidence loss weights $\lambda$ (right).
• Figure 3: Distribution of maximum softmax probability (left) for ID pre-training, (middle) fine-tuning with OE, (right) fine-tuning with DCM. ID and OOD datasets are CIFAR-100 and TinyImageNet, respectively. DCM results in (1) better separation of predictive confidence for ID and OOD inputs, and (2) low predictive confidence on OOD inputs, suggesting that it learns a conservative model.
• Figure 4: Robustness of DCM to hyperparameters.Left: Performance of DCM on a near-OOD detection task (CIFAR-100 [0:50] vs CIFAR-100 [50:100]) with various OOD proportions in the uncertainty dataset. Our methods, DCM-Energy and DCM-Softmax, outperform existing methods across all OOD proportions. Middle: Relative AUROC of DCM with various confidence weights $\lambda$; note the negligible differences in AUROC. Right: Selective classification performance of DCM with uncertainty datasets of various sizes on CIFAR-10 $\rightarrow$ CIFAR-10-C. These plots suggest that DCM is robust to a range of confidence weights and sizes and compositions in the uncertainty dataset.
• Figure 5: Further ablations on robustness of DCM to hyperparameters.Left: Relative AUROC of DCM on 3 regular OOD detection setting, where we vary the number of epochs in the second fine-tuning stage. Our default choice of 10 does not generally achieve the best performance. Middle: Similar to the plot on the left, but we experiment in the challenging near-OOD detection setting. Right: Relative AUROC of DCM where we vary the confidence weight $\lambda$.

Theorems & Definitions (10)

• Proposition 4.1: Lower bound on true confidence
• Proposition 4.2: Low loss implies separation
• Proposition A.1: Lower bound on true confidence
• proof
• Lemma A.2: Pinsker's inequality
• proof
• Lemma A.3: Low loss implies separation, transductive case
• proof
• Proposition A.4: Low loss implies separation
• proof