Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Combating Frequency Simplicity-biased Learning for Domain Generalization

Xilin He, Jingyu Hu, Qinliang Lin, Cheng Luo, Weicheng Xie, Siyang Song, Muhammad Haris Khan, Linlin Shen

TL;DR

Two effective data augmentation modules are proposed designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning.

Abstract

Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. Recent studies show that neural networks frequently suffer from a simplicity-biased learning behavior which leads to over-reliance on specific frequency sets, namely as frequency shortcuts, instead of semantic information, resulting in poor generalization performance. Despite previous data augmentation techniques successfully enhancing generalization performances, they intend to apply more frequency shortcuts, thereby causing hallucinations of generalization improvement. In this paper, we aim to prevent such learning behavior of applying frequency shortcuts from a data-driven perspective. Given the theoretical justification of models' biased learning behavior on different spatial frequency components, which is based on the dataset frequency properties, we argue that the learning behavior on various frequency components could be manipulated by changing the dataset statistical structure in the Fourier domain. Intuitively, as frequency shortcuts are hidden in the dominant and highly dependent frequencies of dataset structure, dynamically perturbating the over-reliance frequency components could prevent the application of frequency shortcuts. To this end, we propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning. Code is available at AdvFrequency (https://github.com/C0notSilly/AdvFrequency).

Towards Combating Frequency Simplicity-biased Learning for Domain Generalization

TL;DR

Two effective data augmentation modules are proposed designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning.

Abstract

Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. Recent studies show that neural networks frequently suffer from a simplicity-biased learning behavior which leads to over-reliance on specific frequency sets, namely as frequency shortcuts, instead of semantic information, resulting in poor generalization performance. Despite previous data augmentation techniques successfully enhancing generalization performances, they intend to apply more frequency shortcuts, thereby causing hallucinations of generalization improvement. In this paper, we aim to prevent such learning behavior of applying frequency shortcuts from a data-driven perspective. Given the theoretical justification of models' biased learning behavior on different spatial frequency components, which is based on the dataset frequency properties, we argue that the learning behavior on various frequency components could be manipulated by changing the dataset statistical structure in the Fourier domain. Intuitively, as frequency shortcuts are hidden in the dominant and highly dependent frequencies of dataset structure, dynamically perturbating the over-reliance frequency components could prevent the application of frequency shortcuts. To this end, we propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning. Code is available at AdvFrequency (https://github.com/C0notSilly/AdvFrequency).

Paper Structure

This paper contains 19 sections, 12 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Analysis
  4. Clarification of the Frequency in this Work
  5. Theoretical Justification
  6. Methodology
  7. Adversarial Amplitude Uncertainty Augmentation (AAUA)
  8. Adversarial Amplitude Dropout (AAD)
  9. Model Training
  10. Experiments
  11. Datasets and Implementation Details
  12. Comparisons with State-of-The-Arts
  13. Frequency Shortcut Evalutation
  14. Empirical Analysis
  15. Frequency sensitivity of trained models
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Illustration of model's frequency simplicity-biased learning and motivation of our method. Deep models tend to learn the simplest solution (e.g. the specific class-wise most distinctive frequency bands) in classification tasks instead of the semantic cues. Our method proposes to adaptively modify the learning difficulty of different frequency components to prevent frequency shortcut learning.
  • Figure 2: Overview of the proposed adversarial frequency augmentation modules.
  • Figure 3: (a) Visualizations of model frequency sensitivity maps on source (Photo) and target domains (Art painting, Cartoon, Sketch). (b) Features manifolds of augmented samples and various domains.