Causality-inspired Latent Feature Augmentation for Single Domain Generalization

TL;DR

A novel causality-inspired latent feature augmentation method for Single-DG by learning the meta-knowledge of feature-level transformation based on causal learning and interventions that can better compensate for the domain-hungry defect and reduce the strong reliance on initial finite image-level transformations and capture more stable domain-invariant causal features for generalization.

Abstract

Single domain generalization (Single-DG) intends to develop a generalizable model with only one single training domain to perform well on other unknown target domains. Under the domain-hungry configuration, how to expand the coverage of source domain and find intrinsic causal features across different distributions is the key to enhancing the models' generalization ability. Existing methods mainly depend on the meticulous design of finite image-level transformation techniques and learning invariant features across domains based on statistical correlation between samples and labels in source domain. This makes it difficult to capture stable semantics between source and target domains, which hinders the improvement of the model's generalization performance. In this paper, we propose a novel causality-inspired latent feature augmentation method for Single-DG by learning the meta-knowledge of feature-level transformation based on causal learning and interventions. Instead of strongly relying on the finite image-level transformation, with the learned meta-knowledge, we can generate diverse implicit feature-level transformations in latent space based on the consistency of causal features and diversity of non-causal features, which can better compensate for the domain-hungry defect and reduce the strong reliance on initial finite image-level transformations and capture more stable domain-invariant causal features for generalization. Extensive experiments on several open-access benchmarks demonstrate the outstanding performance of our model over other state-of-the-art single domain generalization and also multi-source domain generalization methods.

Figures (11)

• Figure 1: Visual examples of ERM and Ours with (a) $16$ and (b) $5$ types of image-level transformation strategies on PACS dataset. ( $\color{green}{\checkmark}$: correct prediction, $\color{red}{\times}$: wrong prediction)
• Figure 2: Causal views on Single-DG. Grey variables represent unobservable variables in practice.
• Figure 3: Schematic of the proposed method. We generate mini-batches of triples in each iteration. Initial features extracted by $F$ is decomposed into causal and non-causal features. Two different "Encoders" learn two types of meta-knowledge about feature-level transformation. With the learned meta-knowledge, we generate diverse implicit feature-level transformations by sampling. Then we perform latent feature augmentation by a shared "Augmentor". $\mathcal{L}_{ind}$ and $\mathcal{L}_{cls}$ encourage the independence and decoupling between latent causal and non-causal features. $\mathcal{L}_{aug}$ encourages the diversity of augmented latent non-causal features and the consistency of augmented latent causal features. $\mathcal{L}_{int}$ is the loss function about causal intervention.
• Figure 4: Feature visualization on PACS. $\text{☆}$, $\triangle$, $\Box$, and $\bigcirc$ denote the features of source domain ("Photo") and other three target domains ("Artpaint", "Cartoon", and "Sketch"). Different colors denote different categories.
• Figure 5: t-SNE visualization of the learned features on target domain "Sketch" (source domain: Photo) of PACS dataset. (a) The learned full features of ERM. (b) The decoupled representation of causal and non-causal features by ours. Non-causal features are presented by '$\times$' in red colors, while the causal features are presented by '$\triangle$' in different colors.