Pseudo-Nonlinear Data Augmentation: A Constrained Energy Minimization Viewpoint
Pingbang Hu, Mahito Sugiyama
TL;DR
The paper targets data augmentation in regimes with limited labels by avoiding training generative models. It proposes a learning-free, energy-based augmentation framework built on information-geometric insights, modeling data as distributions on a curved manifold $\mathcal{S}$ and performing controllable augmentation via forward projection to a base sub-manifold $\mathcal{B}$ and backward projection from a local data sub-manifold $\mathcal{D}$. The approach relies on a log-linear model on posets to encode structured data, and uses dually-flat coordinates $(\theta, \eta)$ to enable efficient convex projections, with many-body sub-manifolds $\mathcal{M}_{\ell}$ controlling augmentation freedom. Empirical results across image, audio, and tabular datasets show competitive or superior performance compared to baselines, along with improved controllability and reduced variance on challenging small datasets. The method offers a scalable, interpretable alternative to learning-based augmentation, applicable broadly across modalities.
Abstract
We propose a simple yet novel data augmentation method for general data modalities based on energy-based modeling and principles from information geometry. Unlike most existing learning-based data augmentation methods, which rely on learning latent representations with generative models, our proposed framework enables an intuitive construction of a geometrically aware latent space that represents the structure of the data itself, supporting efficient and explicit encoding and decoding procedures. We then present and discuss how to design latent spaces that will subsequently control the augmentation with the proposed algorithm. Empirical results demonstrate that our data augmentation method achieves competitive performance in downstream tasks compared to other baselines, while offering fine-grained controllability that is lacking in the existing literature.