Data Augmentation for Deep Learning Regression Tasks by Machine Learning Models

TL;DR

The paper tackles the limited success of deep learning on tabular regression by introducing AutoML-guided data augmentation to generate labeled synthetic data. It presents a two-stage pipeline where an AutoML model labels synthesized samples created from the training data, and a DL model is trained on the augmented set, evaluated across 30 datasets using AutoKeras, H2O, and AutoGluon. The approach yields average improvements exceeding $10\%$, often outperforming existing tabular augmentation methods like C-Mixup and Anchor Data Augmentation, and demonstrates relevance across varying training sizes. This work highlights the practical potential of AutoDL-assisted data synthesis to close the gap between DL and classical ML in tabular regression and points to future directions in AutoDL frameworks and data-augmentation strategies.

Abstract

Deep learning (DL) models have gained prominence in domains such as computer vision and natural language processing but remain underutilized for regression tasks involving tabular data. In these cases, traditional machine learning (ML) models often outperform DL models. In this study, we propose and evaluate various data augmentation (DA) techniques to improve the performance of DL models for tabular data regression tasks. We compare the performance gain of Neural Networks by different DA strategies ranging from a naive method of duplicating existing observations and adding noise to a more sophisticated DA strategy that preserves the underlying statistical relationship in the data. Our analysis demonstrates that the advanced DA method significantly improves DL model performance across multiple datasets and regression tasks, resulting in an average performance increase of over 10\% compared to baseline models without augmentation. The efficacy of these DA strategies was rigorously validated across 30 distinct datasets, with multiple iterations and evaluations using three different automated deep learning (AutoDL) frameworks: AutoKeras, H2O, and AutoGluon. This study demonstrates that by leveraging advanced DA techniques, DL models can realize their full potential in regression tasks, thereby contributing to broader adoption and enhanced performance in practical applications.

Figures (5)

• Figure 1: A schematic view of the method's flow.
• Figure 2: Normalized RMSE values as a function of training size. The figure presents a summary of 50 iterations for ten datasets without augmentation.
• Figure 3: Relative RMSE improvement as a function of training size and number of augmented rows. A summary of ten iterations for the Guo (CS) dataset with varying numbers of augmented rows compared to a baseline without augmentation. The results for all ten datasets are presented in the appendix.
• Figure 4: A breakdown of the improvement by augmentation as a function of the advantage of the TPOT model compared to the original DL model. Positive values on the horizontal axis represent an advantage of the TPOT model compared to each DL; positive values on the vertical axis represent an improvement by the augmentation. Each observation represents the mean of one dataset.
• Figure 5: Relative RMSE improvement as a function of training size and number of augmented rows. A summary of ten iterations for ten datasets with varying numbers of augmented rows compared to a baseline without augmentation.