Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TAROT: Towards Essentially Domain-Invariant Robustness with Theoretical Justification

Dongyoon Yang, Jihu Lee, Yongdai Kim

TL;DR

This paper tackles robust unsupervised domain adaptation under adversarial perturbations by deriving a new generalization bound based on a Robust Margin Disparity Discrepancy and proposing TAROT, a transfer adversarially robust training algorithm. TAROT integrates pseudo-labeling with explicit distribution alignment, leverages a robust pre-trained initialization, and uses a robust-margin–discrepancy objective to learn domain-invariant robust representations. Theoretical results connect target robust risk to source risk, robust discrepancy, and local Lipschitz constants, while empirical results on Office-31, Office-Home, VisDA2017, DomainNet show TAROT achieving state-of-the-art robustness and generalization, including to unseen domains. The work advances both theory and practice in robust domain adaptation by delivering scalable, principled methods with strong performance across diverse domain shifts.

Abstract

Robust domain adaptation against adversarial attacks is a critical research area that aims to develop models capable of maintaining consistent performance across diverse and challenging domains. In this paper, we derive a new generalization bound for robust risk on the target domain using a novel divergence measure specifically designed for robust domain adaptation. Building upon this, we propose a new algorithm named TAROT, which is designed to enhance both domain adaptability and robustness. Through extensive experiments, TAROT not only surpasses state-of-the-art methods in accuracy and robustness but also significantly enhances domain generalization and scalability by effectively learning domain-invariant features. In particular, TAROT achieves superior performance on the challenging DomainNet dataset, demonstrating its ability to learn domain-invariant representations that generalize well across different domains, including unseen ones. These results highlight the broader applicability of our approach in real-world domain adaptation scenarios.

TAROT: Towards Essentially Domain-Invariant Robustness with Theoretical Justification

TL;DR

This paper tackles robust unsupervised domain adaptation under adversarial perturbations by deriving a new generalization bound based on a Robust Margin Disparity Discrepancy and proposing TAROT, a transfer adversarially robust training algorithm. TAROT integrates pseudo-labeling with explicit distribution alignment, leverages a robust pre-trained initialization, and uses a robust-margin–discrepancy objective to learn domain-invariant robust representations. Theoretical results connect target robust risk to source risk, robust discrepancy, and local Lipschitz constants, while empirical results on Office-31, Office-Home, VisDA2017, DomainNet show TAROT achieving state-of-the-art robustness and generalization, including to unseen domains. The work advances both theory and practice in robust domain adaptation by delivering scalable, principled methods with strong performance across diverse domain shifts.

Abstract

Robust domain adaptation against adversarial attacks is a critical research area that aims to develop models capable of maintaining consistent performance across diverse and challenging domains. In this paper, we derive a new generalization bound for robust risk on the target domain using a novel divergence measure specifically designed for robust domain adaptation. Building upon this, we propose a new algorithm named TAROT, which is designed to enhance both domain adaptability and robustness. Through extensive experiments, TAROT not only surpasses state-of-the-art methods in accuracy and robustness but also significantly enhances domain generalization and scalability by effectively learning domain-invariant features. In particular, TAROT achieves superior performance on the challenging DomainNet dataset, demonstrating its ability to learn domain-invariant representations that generalize well across different domains, including unseen ones. These results highlight the broader applicability of our approach in real-world domain adaptation scenarios.
Paper Structure (40 sections, 14 theorems, 62 equations, 5 figures, 14 tables, 1 algorithm)

This paper contains 40 sections, 14 theorems, 62 equations, 5 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Adversarial Training
  4. Unsupervised Domain Adaptation
  5. Related Works
  6. Proposed Method
  7. Robust Margin Disparity Discrepancy
  8. Generalization bound
  9. Two insights
  10. About Local Lipschitz constant
  11. Pseudo labeling only may not be sufficient
  12. Learning Algorithm
  13. Experiments
  14. Performance Evaluation
  15. Training Setup
  16. ...and 25 more sections

Key Result

Proposition 1

Let $\mathcal{S}$ and $\mathcal{T}$ represent the distributions of the source and target domains, respectively. Similarly, let $\mathcal{S}_\mathbf{X}$ and $\mathcal{T}_\mathbf{X}$ denote the marginal distributions of the source and target domains over $\mathbf{X}$, respectively. For every score fun where $f^*=\underset{f\in\mathcal{F}}{\mathop{\mathrm{argmin}}\limits}\{\mathcal{R}_\mathcal{T}^{(\

Figures (5)

  • Figure 1: Overview of TAROT algorithm.
  • Figure 2: Performances of ARTUDA, RFA, SRoUDA, PL and TAROT on DomainNet ($\varepsilon=4/255$). In each cell, the first number is the standard accuracy (%), and the second number is the robust accuracy (%) for AA. Bold numbers indicate the best performance.
  • Figure 3: Performances of ARTUDA, RFA, SRoUDA, PL and TAROT on VisDA2017 ($\varepsilon=8/255$). Standard accuracy (%) / Robust accuracy (%) for AA.
  • Figure 4: Sensitivity Analysis of $\alpha$. $\alpha=0$ corresponds to PL.
  • Figure 5: Effect of Robust-PT across various methods and $\varepsilon$ values. Each plot compares different configurations of PL w/ MDD and TAROT for standard and robust accuracy.

Theorems & Definitions (27)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Definition 3
  • Definition 4: Local Lipschitz Constant
  • Definition 5
  • Definition 6
  • Theorem 1
  • Proposition 2
  • Lemma 1: Lemma C.4 from zhang2019bridging, Theorem 8.1 from mohri2012foundation
  • ...and 17 more