Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feature Augmentation for Self-supervised Contrastive Learning: A Closer Look

Yong Zhang, Rui Zhu, Shifeng Zhang, Xu Zhou, Shifeng Chen, Xiaofan Chen

TL;DR

This work proposes a unified framework to conduct data augmentation in the feature space, known as feature augmentation, which is domain-agnostic, which augments similar features to the original ones and thus improves the data diversity.

Abstract

Self-supervised contrastive learning heavily relies on the view variance brought by data augmentation, so that it can learn a view-invariant pre-trained representation. Beyond increasing the view variance for contrast, this work focuses on improving the diversity of training data, to improve the generalization and robustness of the pre-trained models. To this end, we propose a unified framework to conduct data augmentation in the feature space, known as feature augmentation. This strategy is domain-agnostic, which augments similar features to the original ones and thus improves the data diversity. We perform a systematic investigation of various feature augmentation architectures, the gradient-flow skill, and the relationship between feature augmentation and traditional data augmentation. Our study reveals some practical principles for feature augmentation in self-contrastive learning. By integrating feature augmentation on the instance discrimination or the instance similarity paradigm, we consistently improve the performance of pre-trained feature learning and gain better generalization over the downstream image classification and object detection task.

Feature Augmentation for Self-supervised Contrastive Learning: A Closer Look

TL;DR

This work proposes a unified framework to conduct data augmentation in the feature space, known as feature augmentation, which is domain-agnostic, which augments similar features to the original ones and thus improves the data diversity.

Abstract

Self-supervised contrastive learning heavily relies on the view variance brought by data augmentation, so that it can learn a view-invariant pre-trained representation. Beyond increasing the view variance for contrast, this work focuses on improving the diversity of training data, to improve the generalization and robustness of the pre-trained models. To this end, we propose a unified framework to conduct data augmentation in the feature space, known as feature augmentation. This strategy is domain-agnostic, which augments similar features to the original ones and thus improves the data diversity. We perform a systematic investigation of various feature augmentation architectures, the gradient-flow skill, and the relationship between feature augmentation and traditional data augmentation. Our study reveals some practical principles for feature augmentation in self-contrastive learning. By integrating feature augmentation on the instance discrimination or the instance similarity paradigm, we consistently improve the performance of pre-trained feature learning and gain better generalization over the downstream image classification and object detection task.

Paper Structure

This paper contains 13 sections, 3 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Feature Augmentation for Contrastive Learning: Approach & Exploration Experiments
  4. The Basic Framework with Feature Augmentation
  5. Feature Augmentation Methods
  6. Feature Augmentation Architectures with Predictors
  7. Exploring the Relationship with Data Augmentation
  8. Upgrading BYOL with Feature Augmentation
  9. Experiments: Pre-training & Transferring
  10. Self-supervised Pre-training on ImageNet
  11. Evaluation on ImageNet-1k
  12. Transfer Learning on Object Detection
  13. Conclusion

Figures (3)

  • Figure 1: In self-supervised contrastive learning, data augmentation (DA) introduces view variance usually by transforming the original input. Instead, feature augmentation (FA) aims to further increase the sample diversity in the feature space after the encoder. Feature manipulation (FM) is a special case of FA that focuses on mining hard examples.
  • Figure 2: Contrastive learning architectures with feature augmentation (FA). (a) is the basic framework. (b)-(d) extend (a) with additional predictors.
  • Figure 3: The performance boost of feature augmentation (FA) over baselines when applying different settings of data augmentation (DA). FA makes up a deficiency for DA, and asymmetric DA setting is suitable for FA.