Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty-Guided Alignment for Unsupervised Domain Adaptation in Regression

Ismail Nejjar, Gaetan Frusque, Florent Forest, Olga Fink

TL;DR

UGA is proposed, a novel method that integrates predictive uncertainty into the feature alignment process and fuses information within the embedding space, effectively mitigating issues such as feature collapse in out-of-distribution scenarios.

Abstract

Unsupervised Domain Adaptation for Regression (UDAR) aims to adapt models from a labeled source domain to an unlabeled target domain for regression tasks. Traditional feature alignment methods, successful in classification, often prove ineffective for regression due to the correlated nature of regression features. To address this challenge, we propose Uncertainty-Guided Alignment (UGA), a novel method that integrates predictive uncertainty into the feature alignment process. UGA employs Evidential Deep Learning to predict both target values and their associated uncertainties. This uncertainty information guides the alignment process and fuses information within the embedding space, effectively mitigating issues such as feature collapse in out-of-distribution scenarios. We evaluate UGA on two computer vision benchmarks and a real-world battery state-of-charge prediction across different manufacturers and operating temperatures. Across 52 transfer tasks, UGA on average outperforms existing state-of-the-art methods. Our approach not only improves adaptation performance but also provides well-calibrated uncertainty estimates.

Uncertainty-Guided Alignment for Unsupervised Domain Adaptation in Regression

TL;DR

UGA is proposed, a novel method that integrates predictive uncertainty into the feature alignment process and fuses information within the embedding space, effectively mitigating issues such as feature collapse in out-of-distribution scenarios.

Abstract

Unsupervised Domain Adaptation for Regression (UDAR) aims to adapt models from a labeled source domain to an unlabeled target domain for regression tasks. Traditional feature alignment methods, successful in classification, often prove ineffective for regression due to the correlated nature of regression features. To address this challenge, we propose Uncertainty-Guided Alignment (UGA), a novel method that integrates predictive uncertainty into the feature alignment process. UGA employs Evidential Deep Learning to predict both target values and their associated uncertainties. This uncertainty information guides the alignment process and fuses information within the embedding space, effectively mitigating issues such as feature collapse in out-of-distribution scenarios. We evaluate UGA on two computer vision benchmarks and a real-world battery state-of-charge prediction across different manufacturers and operating temperatures. Across 52 transfer tasks, UGA on average outperforms existing state-of-the-art methods. Our approach not only improves adaptation performance but also provides well-calibrated uncertainty estimates.
Paper Structure (24 sections, 1 theorem, 10 equations, 26 figures, 6 tables)

This paper contains 24 sections, 1 theorem, 10 equations, 26 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Unsupervised Domain Adaptation
  4. Uncertainty-guided domain adaptation
  5. Preliminaries
  6. Methods
  7. Problem setting
  8. Theoretical Motivation
  9. Overview of the proposed method
  10. Uncertainty-Guided Feature Alignment
  11. Uncertainty-Guided Posterior Alignment
  12. Experiments
  13. Datasets and Tasks
  14. Computer Vision Benchmarks
  15. PHM task: State-of-Charge prediction
  16. ...and 9 more sections

Key Result

Theorem 1

Let $g(\cdot)$ be a deterministic feature extractor, and let $\mathbf{z}^S = g(x^S)$, $\mathbf{z}^T = g(x^T)$ denote the extracted features for the source and target domains, respectively. Assuming that $y^S|\mathbf{z}^S$ and $y^T|\mathbf{z}^T$ follow Gaussian distributions, if the features are perf where $\ell$ denotes the mean squared error loss, $p_S$ and $p_T$ are the source and target domain

Figures (26)

  • Figure 2: Overview of our proposed Uncertainty-Guided Alignment (UGA) framework for unsupervised domain adaptation in regression. UGA leverages the Deep Evidential Regression (DER) uncertainty framework. We propose aligning augmented feature representations with uncertainty to guide traditional feature alignment. We also introduce Posterior alignment, an approximation method aligning the evidential parameters.
  • Figure 5: Example of State of Charge (SoC) curves and associated features (voltage, current, temperature, and time) for LG batteries that have been charged and discharged until reaching 0% capacity. The plots illustrate the impact of temperature on battery capacity, discharge rate, and overall performance throughout the battery's lifecycle.
  • Figure 6: Comparison of SoC curves and associated features (voltage, current, temperature, and time) for Panasonic batteries that have been charged and discharged until reaching 0% capacity. The plots highlight the differences in battery performance between the two manufacturers, including variations in capacity, discharge rate, and temperature sensitivity throughout the battery's lifecycle.
  • Figure : (a) Density plot comparing model predictions for battery State of charge between the source domain at -20°C and target domains of increasing temperature, showing the effect of domain shift on prediction patterns.
  • Figure : (a) Noise
  • ...and 21 more figures

Theorems & Definitions (2)

  • Definition 4.1: Adaptation Gap
  • Theorem 1