Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Expansion of Small Datasets: An Expansive Graph Approach

Vahid Jebraeeli, Bo Jiang, Hamid Krim, Derya Cansever

TL;DR

An Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples, which leverages neural networks' non-linear latent space, captured by a Koopman operator, to create a linear feature space for dataset expansion.

Abstract

Limited data availability in machine learning significantly impacts performance and generalization. Traditional augmentation methods enhance moderately sufficient datasets. GANs struggle with convergence when generating diverse samples. Diffusion models, while effective, have high computational costs. We introduce an Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples. It uses expander graph mappings and feature interpolation to preserve data distribution and feature relationships. The model leverages neural networks' non-linear latent space, captured by a Koopman operator, to create a linear feature space for dataset expansion. An autoencoder with self-attention layers and optimal transport refines distributional consistency. We validate by comparing classifiers trained on generated data to those trained on original datasets. Results show comparable performance, demonstrating the model's potential to augment training data effectively. This work advances data generation, addressing scarcity in machine learning applications.

Generative Expansion of Small Datasets: An Expansive Graph Approach

TL;DR

An Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples, which leverages neural networks' non-linear latent space, captured by a Koopman operator, to create a linear feature space for dataset expansion.

Abstract

Limited data availability in machine learning significantly impacts performance and generalization. Traditional augmentation methods enhance moderately sufficient datasets. GANs struggle with convergence when generating diverse samples. Diffusion models, while effective, have high computational costs. We introduce an Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples. It uses expander graph mappings and feature interpolation to preserve data distribution and feature relationships. The model leverages neural networks' non-linear latent space, captured by a Koopman operator, to create a linear feature space for dataset expansion. An autoencoder with self-attention layers and optimal transport refines distributional consistency. We validate by comparing classifiers trained on generated data to those trained on original datasets. Results show comparable performance, demonstrating the model's potential to augment training data effectively. This work advances data generation, addressing scarcity in machine learning applications.
Paper Structure (20 sections, 9 equations, 3 figures, 2 tables, 1 algorithm)

This paper contains 20 sections, 9 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Background
  3. Koopman Feature Space
  4. Data Distribution Preservation
  5. Methodology
  6. Autoencoder:Koopman Feature Space
  7. Feature Refinement: A Self-Attention Mechanism
  8. Expander Graph Mapping
  9. Loss Objective Functions and Regularization
  10. Total Loss ($\mathcal{L}_{\text{total}}$):
  11. Reconstruction Loss ($\mathcal{L}_{re}$):
  12. Distributional Consistency($\mathcal{L}_{\mathcal{W}}$):
  13. Classification Loss ($\mathcal{L}_{ce}$):
  14. Expansion Diversity Loss ($\mathcal{L}_{cov}$):
  15. Experiments and Results
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Overall architecture of Expansive Synthesis model. (a) Pretraining phase using a similar larger dataset $X"$ to learn general features, which are encoded and decoded to produce $Y"$. (b) Fine-tuning phase on the smaller minimal sample dataset $X$ to adapt the model's weights, followed by the expansion of $X$ to generate the synthesized dataset $X'$ using expander graph mapping and self-attention mechanisms. The expanded dataset $X'$ is then used to train a classifier.
  • Figure 2: Architecture of Expander Graph Mapping
  • Figure 3: Stages of Implementation and evaluation of a expansion model