Unsupervised domain adaptation by learning using privileged information

TL;DR

DALUPI tackles unsupervised domain adaptation under limited input overlap by leveraging privileged information (PI) available only during training. It develops a two-stage estimator h = g \circ f and a practical end-to-end ROI-based variant to exploit PI types such as binary attributes and regions of interest in image data, with theoretical identifiability of the target risk and a PAC-style generalization perspective. The approach yields empirical gains across CelebA, a synthetic digit task, MS-COCO, and Chest X-ray classification, demonstrating improved adaptation, robustness to spurious source correlations, and enhanced sample efficiency, especially in small-sample regimes. These results underscore the practical value of training-time PI for domain transfer and motivate broader data collection of PI-annotated examples for robust UDA.

Abstract

Successful unsupervised domain adaptation is guaranteed only under strong assumptions such as covariate shift and overlap between input domains. The latter is often violated in high-dimensional applications like image classification which, despite this limitation, continues to serve as inspiration and benchmark for algorithm development. In this work, we show that training-time access to side information in the form of auxiliary variables can help relax restrictions on input variables and increase the sample efficiency of learning at the cost of collecting a richer variable set. As this information is assumed available only during training, not in deployment, we call this problem unsupervised domain adaptation by learning using privileged information (DALUPI). To solve this problem, we propose a simple two-stage learning algorithm, inspired by our analysis of the expected error in the target domain, and a practical end-to-end variant for image classification. We propose three evaluation tasks based on classification of entities in photos and anomalies in medical images with different types of available privileged information (binary attributes and single or multiple regions of interest). We demonstrate across these tasks that using privileged information in learning can reduce errors in domain transfer compared to baselines, be robust to spurious correlations in the source domain, and increase sample efficiency.

Figures (9)

• Figure 1: Examples of domain adaptation tasks with different types of privileged information (PI). During training, input samples $X$ and PI $W$ are drawn from both source and target domains. Labels $Y$ are only available from the source domain. At test time, a target sample $X$ is observed. We consider three types of PI: binary attribute vectors, a single region of interest, and multiple regions of interest.
• Figure 2: An illustration of domain overlap being more plausible when we consider appropriate forms of privileged information $W$, such as a region of interest of an image. Source and target domains $\mathcal{S}, \mathcal{T}$ are here indoor and outdoor images $X$ and the task is to identify the animal $Y$ in the image.
• Figure 3: Digit classification. Target domain accuracy as a function of association $\epsilon$ between background and label in $\mathcal{S}$. As the skew increases, the target-domain performance of the non-privileged models deteriorates.
• Figure 4: Entity classification. Target domain AUC. The performance of SL-S and SL-T is extended across the x-axes for visual purposes. DANN and MDD use an increasing fraction of target samples $\tilde{x}$ but no PI.
• Figure 5: Left: Example from the X-ray target test set with label CM. The red rectangle indicates the bounding box predicted by DALUPI. Right: saliency map for CM for SL-S.

Theorems & Definitions (5)

• Proposition 1
• Proposition 2
• Proposition
• proof
• proof