Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReTaSA: A Nonparametric Functional Estimation Approach for Addressing Continuous Target Shift

Hwanwoo Kim, Xin Zhang, Jiwei Zhao, Qinglong Tian

TL;DR

The paper tackles continuous target shift in regression under unsupervised domain adaptation by recasting the problem as estimating a continuous importance weight $\omega(y)=p_t(y)/p_s(y)$ from an ill-posed Fredholm integral equation. It develops ReTaSA, a nonparametric regularization framework using Tikhonov regularization to stabilize the solution, with a kernel-density-estimation–based data-driven implementation that yields a simple linear system to solve for $\rho(y)=\omega(y)-1$ and thus $\omega(y)$. The authors establish identifiability under a completeness condition, prove consistency and convergence rates for the estimator in a weighted $L^2$ sense, and demonstrate superior performance over baselines on synthetic and real regression datasets. The work provides theoretical guarantees and a practical, computation-friendly method to adapt regression models to target-domain shifts without requiring distribution matching, with extensions to high-dimensional features via black-box mappings. This advances robust regression under distributional shifts by bridging nonparametric inverse problems with domain adaptation theory.

Abstract

The presence of distribution shifts poses a significant challenge for deploying modern machine learning models in real-world applications. This work focuses on the target shift problem in a regression setting (Zhang et al., 2013; Nguyen et al., 2016). More specifically, the target variable y (also known as the response variable), which is continuous, has different marginal distributions in the training source and testing domain, while the conditional distribution of features x given y remains the same. While most literature focuses on classification tasks with finite target space, the regression problem has an infinite dimensional target space, which makes many of the existing methods inapplicable. In this work, we show that the continuous target shift problem can be addressed by estimating the importance weight function from an ill-posed integral equation. We propose a nonparametric regularized approach named ReTaSA to solve the ill-posed integral equation and provide theoretical justification for the estimated importance weight function. The effectiveness of the proposed method has been demonstrated with extensive numerical studies on synthetic and real-world datasets.

ReTaSA: A Nonparametric Functional Estimation Approach for Addressing Continuous Target Shift

TL;DR

The paper tackles continuous target shift in regression under unsupervised domain adaptation by recasting the problem as estimating a continuous importance weight from an ill-posed Fredholm integral equation. It develops ReTaSA, a nonparametric regularization framework using Tikhonov regularization to stabilize the solution, with a kernel-density-estimation–based data-driven implementation that yields a simple linear system to solve for and thus . The authors establish identifiability under a completeness condition, prove consistency and convergence rates for the estimator in a weighted sense, and demonstrate superior performance over baselines on synthetic and real regression datasets. The work provides theoretical guarantees and a practical, computation-friendly method to adapt regression models to target-domain shifts without requiring distribution matching, with extensions to high-dimensional features via black-box mappings. This advances robust regression under distributional shifts by bridging nonparametric inverse problems with domain adaptation theory.

Abstract

The presence of distribution shifts poses a significant challenge for deploying modern machine learning models in real-world applications. This work focuses on the target shift problem in a regression setting (Zhang et al., 2013; Nguyen et al., 2016). More specifically, the target variable y (also known as the response variable), which is continuous, has different marginal distributions in the training source and testing domain, while the conditional distribution of features x given y remains the same. While most literature focuses on classification tasks with finite target space, the regression problem has an infinite dimensional target space, which makes many of the existing methods inapplicable. In this work, we show that the continuous target shift problem can be addressed by estimating the importance weight function from an ill-posed integral equation. We propose a nonparametric regularized approach named ReTaSA to solve the ill-posed integral equation and provide theoretical justification for the estimated importance weight function. The effectiveness of the proposed method has been demonstrated with extensive numerical studies on synthetic and real-world datasets.
Paper Structure (14 sections, 1 theorem, 11 equations, 2 tables)

This paper contains 14 sections, 1 theorem, 11 equations, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Our Contributions
  4. Methodology
  5. Operator Notations
  6. Identifiability
  7. Ill-posedness & Regularization
  8. Implementation
  9. Theoretical Results
  10. Numerical Studies
  11. Experiment with Multivariate Regression
  12. Discussion
  13. Proofs
  14. Additional Numerical Results

Key Result

Theorem 1

For $\beta \ge 2$, under Assumptions ASSUMP:HS - ASSUMP:bandwidth, one can show that $||\widehat{\rho}_{\alpha} - \rho_0 ||^2$ is of order In particular, if $\max\left(h^{2\gamma}, \sqrt{\frac{{\rm log} (m)}{m h^p}}h^\gamma, \frac{{\rm log} (m)}{m h^p}\right) = o_p(\alpha^2)~\text{and} ~\lim_{\substack{\alpha \to 0 \\ n \to \infty}} n\alpha^2 = \infty$, $\widehat{\rho}_{\alpha}$ converges in prob

Theorems & Definitions (10)

  • Remark 1
  • definition 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Theorem 1
  • Remark 6
  • Remark 7
  • Remark 8