Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dataset Augmentation in Feature Space

Terrance DeVries, Graham W. Taylor

TL;DR

This work proposes a domain-agnostic data augmentation framework that operates in a learned feature space rather than input space by training a sequence autoencoder to produce context vectors. It applies noise, interpolation, and extrapolation within this space, with extrapolation emerging as the most effective transformation across multiple domains (speech, motion, sign language, and images). The authors show that extrapolation often yields substantial performance gains, while interpolation can degrade performance on some datasets, and demonstrate compatibility with high-capacity architectures when decoding back to input space. Overall, the approach offers a simple, generalizable augmentation strategy that complements traditional domain-specific methods and improves generalization under limited labeled data.

Abstract

Dataset augmentation, the practice of applying a wide array of domain-specific transformations to synthetically expand a training set, is a standard tool in supervised learning. While effective in tasks such as visual recognition, the set of transformations must be carefully designed, implemented, and tested for every new domain, limiting its re-use and generality. In this paper, we adopt a simpler, domain-agnostic approach to dataset augmentation. We start with existing data points and apply simple transformations such as adding noise, interpolating, or extrapolating between them. Our main insight is to perform the transformation not in input space, but in a learned feature space. A re-kindling of interest in unsupervised representation learning makes this technique timely and more effective. It is a simple proposal, but to-date one that has not been tested empirically. Working in the space of context vectors generated by sequence-to-sequence models, we demonstrate a technique that is effective for both static and sequential data.

Dataset Augmentation in Feature Space

TL;DR

This work proposes a domain-agnostic data augmentation framework that operates in a learned feature space rather than input space by training a sequence autoencoder to produce context vectors. It applies noise, interpolation, and extrapolation within this space, with extrapolation emerging as the most effective transformation across multiple domains (speech, motion, sign language, and images). The authors show that extrapolation often yields substantial performance gains, while interpolation can degrade performance on some datasets, and demonstrate compatibility with high-capacity architectures when decoding back to input space. Overall, the approach offers a simple, generalizable augmentation strategy that complements traditional domain-specific methods and improves generalization under limited labeled data.

Abstract

Dataset augmentation, the practice of applying a wide array of domain-specific transformations to synthetically expand a training set, is a standard tool in supervised learning. While effective in tasks such as visual recognition, the set of transformations must be carefully designed, implemented, and tested for every new domain, limiting its re-use and generality. In this paper, we adopt a simpler, domain-agnostic approach to dataset augmentation. We start with existing data points and apply simple transformations such as adding noise, interpolating, or extrapolating between them. Our main insight is to perform the transformation not in input space, but in a learned feature space. A re-kindling of interest in unsupervised representation learning makes this technique timely and more effective. It is a simple proposal, but to-date one that has not been tested empirically. Working in the space of context vectors generated by sequence-to-sequence models, we demonstrate a technique that is effective for both static and sequential data.

Paper Structure

This paper contains 15 sections, 5 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model
  4. Sequence Autoencoder
  5. Augmentation in Feature Space
  6. Experiments
  7. Visualization - Sinusoids
  8. Visualization - UJI Pen Characters
  9. Spoken Arabic Digits
  10. Australian Sign Language Signs (AUSLAN)
  11. UCFKinect
  12. Image Classification: MNIST and CIFAR-10
  13. Classification on Reconstructed Images
  14. Discussion
  15. Conclusion

Figures (3)

  • Figure 1: System architecture composed of three steps. (a) A sequence autoencoder learns a feature space from unlabeled data, representing each sequence by a context vector ($C$). (b) Data is encoded to context vectors and augmented by adding noise, interpolating, or extrapolating (here we depict interpolation). (c) The resulting context vectors can either be used directly as features for supervised learning with a static classifier, or they can be decoded to reconstruct full sequences for training a sequence classifier.
  • Figure 2: Sinusoids with various transformations applied in feature space. (a) Random noise added with $\gamma = 0.5$. (b) Interpolation between two sinusoids for values of $\lambda$ between 0 and 1. (c) Extrapolation between two sinusoids for values of $\lambda$ between 0 and 1. Best viewed in colour.
  • Figure 3: Interpolation (a) and extrapolation (b) between handwritten characters. Character (0,i) is interpolated/extrapolated with character (j,0) to form character (i,j), where i is the row number and j is the column number. Original characters are shown in bold.