StyDeSty: Min-Max Stylization and Destylization for Single Domain Generalization

TL;DR

This work tackles single domain generalization by introducing StyDeSty, a framework that jointly learns stylization and destylization to form a latent, style-invariant domain $\widetilde{\mathbb{X}}$ from a single source domain. The stylization module generates diverse stylized samples, while the destylization module maps both source and stylized features to the latent domain using AdaIN, with a perceptual alignment constraint to preserve content. A neural architecture search (NAS) automatically locates the optimal position for the destylization layer within the backbone, and a min-max training regime ties stylization, destylization, and task learning together. Experiments on Digits, CIFAR-10-C, PACS, and depth estimation show that StyDeSty achieves up to 13.44% improvements over state-of-the-art single DG methods and demonstrates versatility across classification and regression tasks. The approach provides a principled, plug-in mechanism to improve robustness to unseen domain styles in real-world applications.

Abstract

Single domain generalization (single DG) aims at learning a robust model generalizable to unseen domains from only one training domain, making it a highly ambitious and challenging task. State-of-the-art approaches have mostly relied on data augmentations, such as adversarial perturbation and style enhancement, to synthesize new data and thus increase robustness. Nevertheless, they have largely overlooked the underlying coherence between the augmented domains, which in turn leads to inferior results in real-world scenarios. In this paper, we propose a simple yet effective scheme, termed as \emph{StyDeSty}, to explicitly account for the alignment of the source and pseudo domains in the process of data augmentation, enabling them to interact with each other in a self-consistent manner and further giving rise to a latent domain with strong generalization power. The heart of StyDeSty lies in the interaction between a \emph{stylization} module for generating novel stylized samples using the source domain, and a \emph{destylization} module for transferring stylized and source samples to a latent domain to learn content-invariant features. The stylization and destylization modules work adversarially and reinforce each other. During inference, the destylization module transforms the input sample with an arbitrary style shift to the latent domain, in which the downstream tasks are carried out. Specifically, the location of the destylization layer within the backbone network is determined by a dedicated neural architecture search (NAS) strategy. We evaluate StyDeSty on multiple benchmarks and demonstrate that it yields encouraging results, outperforming the state of the art by up to {13.44%} on classification accuracy. Codes are available here: https://github.com/Huage001/StyDeSty.

Figures (11)

• Figure 1: Workflows of general DG solution, existing single DG solution, and the proposed solution. (a) General DG methods are trained on multiple source domains to learn domain-invariant representations for generalization. (b) Existing single DG methods typically leverage data augmentation techniques to increase the domain diversity and then conduct training directly on the pseudo domains. (c) The proposed single DG solution enables an explicit stylization and destylization mechanisms to learn a latent domain, where the downstream tasks are performed. The stylization and destylization work in an adversarial fashion, and the location of destylization is determined by a NAS algorithm.
• Figure 2: StyDeSty framework consists of a stylization module $G$, a destyliation module $F$, and a task head $H$, where a NAS algorithm is involved to search an optimal position of the AdaIN layer for destylization. Black and red arrows denote forward pass and loss computation and IN represents instance normalization.
• Figure 3: Left: TSNE visualizations of stylized and original source samples for by different fashions of destylization. Right: Visualizations of task loss and alignment loss with destylization at different locations in a deep network.
• Figure 4: Relationships between accuracy results and corruption levels of four categories on CIFAR-10-C dataset. Our method demonstrates more robustness compared with other methods as the corruption increases.
• Figure 5: Sensitivity analysis for loss weights: $\alpha$, $\beta$, $\lambda$, and $B$.