Entropy Reweighted Conformal Classification

TL;DR

This work tackles the inefficiency of conformal prediction when coupled with confidence calibration by introducing entropy-based reweighting of the predicted distribution. The method uses the classifier entropy $H(X)$ with a tunable temperature $T$ to reweight logits $z_k(X)$ and obtain $ ilde{f}(X)$, from which conformity scores $ ilde{A}_n$ are computed to form prediction sets that balance coverage and size. A temperature search over $T$ selects $T^*$ to minimize the average prediction-set size, yielding an adaptive CP scheme with improved efficiency while maintaining coverage guarantees. Empirical results on datasets including AG News, CARER, MNIST, and Fashion MNIST demonstrate competitive coverage and reduced set sizes compared to baselines, highlighting the practical impact for calibrated uncertainty quantification in diverse domains.

Abstract

Conformal Prediction (CP) is a powerful framework for constructing prediction sets with guaranteed coverage. However, recent studies have shown that integrating confidence calibration with CP can lead to a degradation in efficiency. In this paper, We propose an adaptive approach that considers the classifier's uncertainty and employs entropy-based reweighting to enhance the efficiency of prediction sets for conformal classification. Our experimental results demonstrate that this method significantly improves efficiency.

Figures (1)

• Figure 1: Size vs. Coverage plots for different datasets and score functions.