Conformal Prediction Under Generalized Covariate Shift with Posterior Drift

TL;DR

This work develops a weighted conformal prediction framework for multiclass classification under generalized covariate shift with posterior drift (g-CSPD). By leveraging both source and target labeled data, the proposed WCC-CSPD method estimates class posteriors from the source and calibrates thresholds using a combined calibration set, with weights computed via Radon-Nikodym derivatives and accelerated by Newton's identities. The authors establish a target-domain coverage guarantee $\mathbb{Q}(Y_T\in\hat{C}(X_T)) \ge 1-\alpha$ under CSPD or g-CSPD, and prove a convergence rate for the calibration error that improves with larger calibration samples that include target data. Empirical results on simulations and semi-synthetic maternal health data demonstrate robust target-coverage performance under distribution shifts, outperforming standard weighted conformal prediction and split conformal baselines while maintaining efficient prediction-set lengths. Overall, the paper provides a practical, theoretically-justified approach for reliable uncertainty quantification under transfer learning with covariate and posterior drift.

Abstract

In many real applications of statistical learning, collecting sufficiently many training data is often expensive, time-consuming, or even unrealistic. In this case, a transfer learning approach, which aims to leverage knowledge from a related source domain to improve the learning performance in the target domain, is more beneficial. There have been many transfer learning methods developed under various distributional assumptions. In this article, we study a particular type of classification problem, called conformal prediction, under a new distributional assumption for transfer learning. Classifiers under the conformal prediction framework predict a set of plausible labels instead of one single label for each data instance, affording a more cautious and safer decision. We consider a generalization of the \textit{covariate shift with posterior drift} setting for transfer learning. Under this setting, we propose a weighted conformal classifier that leverages both the source and target samples, with a coverage guarantee in the target domain. Theoretical studies demonstrate favorable asymptotic properties. Numerical studies further illustrate the usefulness of the proposed method.

Figures (5)

• Figure 1: Illustration of $\textbf{g-CSPD}$ at $\alpha$
• Figure 2: Performance of all baselines in the simulation setup described in Section \ref{['sec:simulation']}. The blue vertical lines correspond to the target coverage level ($1 - \alpha = 0.9$). The average marginal coverage rates (over all $r$) are: WCC-CSPD (oracle weight, estimated weight) at 0.932, 0.903; WCP (oracle weight, estimated weight) at 0.869, 0.811; CP at 0.885.
• Figure 3: Performance of all baselines in the semi-synthetic setup described in Section \ref{['sec:simulation']}. In (a), the red dashed line illustrates that $\textbf{g-CSPD}$ holds at the corresponding level. The marginal coverage rates are: WCC-CSPD at 0.898, WCP at 0.834, CP at 0.854.
• Figure 4: An illustration of the simulation study when $r = 2$.
• Figure 5: An illustration for one term in the weight calculation under a specific permutation

Theorems & Definitions (15)

• Lemma 1
• Theorem 1
• Lemma 2
• Definition 3.1: $\textbf{g-CSPD}$ at $\alpha$
• Theorem 2
• Lemma 3: tibshirani2019conformal
• Lemma 1
• proof
• Theorem 1
• proof