Towards Backwards-Compatible Data with Confounded Domain Adaptation

TL;DR

This paper presents a novel framework for domain adaptation, based on minimizing the expected divergence between the source and target conditional distributions, conditioning on possible confounders, which seeks to achieve general-purpose data backwards compatibility.

Abstract

Most current domain adaptation methods address either covariate shift or label shift, but are not applicable where they occur simultaneously and are confounded with each other. Domain adaptation approaches which do account for such confounding are designed to adapt covariates to optimally predict a particular label whose shift is confounded with covariate shift. In this paper, we instead seek to achieve general-purpose data backwards compatibility. This would allow the adapted covariates to be used for a variety of downstream problems, including on pre-existing prediction models and on data analytics tasks. To do this we consider a modification of generalized label shift (GLS), which we call confounded shift. We present a novel framework for this problem, based on minimizing the expected divergence between the source and target conditional distributions, conditioning on possible confounders. Within this framework, we provide concrete implementations using the Gaussian reverse Kullback-Leibler divergence and the maximum mean discrepancy. Finally, we demonstrate our approach on synthetic and real datasets.

Figures (24)

• Figure 1: Diagram depicting our motivating scenario. There is confounded shift between source (V1) and target (V2) domains. The different shades of blue of the EEG data portray the "covariate shift" between V1 features and V2 features. The seizure confounding variable differs in distribution between source and target; we portray this "label shift" in seizure status proportions ($\times$ vs $+$) between V1 and V2. With V1 training data, we had previously learned prediction models for seizure and depression given V1 features. With V2 training data, we learn a mapping from V2 features to V1 features via ConDo. At V2 test time, neither seizure nor depression status are provided, but we combine the ConDo V2-to-V1 mapping with the previously-learned prediction models. The EEG data correspond to the features variable $X$, seizure status corresponds to the confounding variable $Z$, and depression status corresponds to the downstream prediction variable $Y$.
• Figure 2: ConDo methods are superior to Gaussian OT when confounded label shift and feature shift are present. The columns, in order, correspond to a confounder with a linear homoscedastic effect, a confounder with a linear heteroscedastic effect, and a confounder with a nonlinear heteroscedastic effect. The first row depicts the problem setup, while the remaining rows depict the performance of Gaussian OT and our ConDo methods. Red points overlapping with green points is indicative of high accuracy. In each subplot, we provide the rMSE on training source data (depicted), and in parentheses, the rMSE on heldout source data (not depicted) generated with confounder sampled from target prior $\mathcal{D}^Z_T$. The printed rMSEs are averaged over 10 independent random simulation runs, while the plots depict the results from the final simulation run.
• Figure 3: Test errors for experiment with synthetic 1d features with 1d continuous confounder. For each adaptation method, we compute the rMSE of true target feature values vs inferred target feature values after adaptation, then average over 10 simulations.
• Figure 4: Results for transforming 1d data with multiple continuous confounders, with extra irrelevant $\mathcal{N}(0, 1)$ confounders, shown in (A), and with noisy additive decomposition, shown in (B). The rMSEs are averaged over 10 random simulations are shown for heldout test data (100 samples per simulation). The columns, in order, correspond to a confounder with a linear homoscedastic effect, a confounder with a linear heteroscedastic effect, and a confounder with a nonlinear heteroscedastic effect.
• Figure 5: Results for transforming 1d data with a 1d categorical confounder. (A) Plot of rMSE vs sample size for each of the domain adaptation methods. Each rMSE was averaged over 10 simulations, with the vertical lines indicating 1 standard deviation over the simulations. (B) Plot of runtime vs sample size for each of the domain adaptation methods.