Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalizing to Unseen Domains: A Survey on Domain Generalization

Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, Philip S. Yu

TL;DR

This survey comprehensively maps the domain generalization landscape, formalizing the problem, summarizing key theory, and organizing methods into data manipulation, representation learning, and learning strategies. It connects DG with related fields through formal bounds and discusses practical considerations, benchmark datasets, and a public codebase (DeepDG) to standardize evaluation. The work highlights open challenges including continuous and open-domain DG, interpretability, and test-time generalization, offering a roadmap for future research and application across vision, NLP, and healthcare. By synthesizing theory, algorithms, and benchmarks, it provides researchers with concrete directions to build models that robustly generalize to unseen domains.

Abstract

Machine learning systems generally assume that the training and testing distributions are the same. To this end, a key requirement is to develop models that can generalize to unseen distributions. Domain generalization (DG), i.e., out-of-distribution generalization, has attracted increasing interests in recent years. Domain generalization deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain. Great progress has been made in the area of domain generalization for years. This paper presents the first review of recent advances in this area. First, we provide a formal definition of domain generalization and discuss several related fields. We then thoroughly review the theories related to domain generalization and carefully analyze the theory behind generalization. We categorize recent algorithms into three classes: data manipulation, representation learning, and learning strategy, and present several popular algorithms in detail for each category. Third, we introduce the commonly used datasets, applications, and our open-sourced codebase for fair evaluation. Finally, we summarize existing literature and present some potential research topics for the future.

Generalizing to Unseen Domains: A Survey on Domain Generalization

TL;DR

This survey comprehensively maps the domain generalization landscape, formalizing the problem, summarizing key theory, and organizing methods into data manipulation, representation learning, and learning strategies. It connects DG with related fields through formal bounds and discusses practical considerations, benchmark datasets, and a public codebase (DeepDG) to standardize evaluation. The work highlights open challenges including continuous and open-domain DG, interpretability, and test-time generalization, offering a roadmap for future research and application across vision, NLP, and healthcare. By synthesizing theory, algorithms, and benchmarks, it provides researchers with concrete directions to build models that robustly generalize to unseen domains.

Abstract

Machine learning systems generally assume that the training and testing distributions are the same. To this end, a key requirement is to develop models that can generalize to unseen distributions. Domain generalization (DG), i.e., out-of-distribution generalization, has attracted increasing interests in recent years. Domain generalization deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain. Great progress has been made in the area of domain generalization for years. This paper presents the first review of recent advances in this area. First, we provide a formal definition of domain generalization and discuss several related fields. We then thoroughly review the theories related to domain generalization and carefully analyze the theory behind generalization. We categorize recent algorithms into three classes: data manipulation, representation learning, and learning strategy, and present several popular algorithms in detail for each category. Third, we introduce the commonly used datasets, applications, and our open-sourced codebase for fair evaluation. Finally, we summarize existing literature and present some potential research topics for the future.

Paper Structure

This paper contains 52 sections, 3 theorems, 17 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Formalization of Domain Generalization
  4. Related Research Areas
  5. Theory
  6. Domain Adaptation
  7. Domain Generalization
  8. Average risk estimation error bound
  9. Generalization risk bound
  10. Methodology
  11. Data Manipulation
  12. Data augmentation-based DG
  13. Domain randomization
  14. Adversarial data augmentation
  15. Data generation-based DG
  16. ...and 37 more sections

Key Result

Theorem 1

Let $d$ be the Vapnik–Chervonenkis (VC) dimension vapnik1994measuring of $\mathcal{H}$, and $\mathcal{U}^s$ and $\mathcal{U}^t$ be unlabeled samples of size $n$ from the two domains. Then for any $h \in \mathcal{H}$ and $\delta \in (0,1)$, the following inequality holds with probability at least $ where ${\hat{d}}_{\mathcal{H}\Delta\mathcal{H}}(\mathcal{U}^s, \mathcal{U}^t)$ is the estimate of $

Figures (4)

  • Figure 1: Examples from the dataset PACS li2017deeper for domain generalization. The training set is composed of images belonging to domains of sketch, cartoon, and art paintings. DG aims to learn a generalized model that performs well on the unseen target domain of photos.
  • Figure 2: Illustration of domain generalization. Adapted from blanchard2011generalizing.
  • Figure 3: Taxonomy of domain generalization methods.
  • Figure 4: Several applications of domain generalization.

Theorems & Definitions (5)

  • Definition 1: Domain
  • Definition 2: Domain generalization
  • Theorem 1: Domain adaptation error bound (non-asymptotic) ben2010theory (Thm. 2)
  • Theorem 2: Average risk estimation error bound for binary classification blanchard2011generalizing
  • Theorem 3: Domain generalization error bound albuquerque2019adversarial